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PREFACE 


This  text,  and  the  Bureau  of  Ships  courses  that  use  it,  re¬ 
presents  a  substantial  departure  from  the  content  of  Pc lia¬ 
bility,  Maintainability,  and  System  Effectiveness  texts  and 
courses  currently  available.  The  departure  is  necessitated  by 
specific  BuSh ips  management  and  technical  needs,  by  significant 
omissions  in  previously  available  material,  and  by  the  current 
dynamic  growth  of  the  technology.  Here  are  the  principal 
cons ideratiens : 

1.  The  point  of  view  and  language  is  for  those  who  deal  with 
contractors,  as  well  as  those  in  BuShips  who  must  resign 
for  the  required  reliability  and  maintainability. 

2.  The  text  fully  recognizes  the  current  limitations  or  the 
"MTBF"  approach,  particularly  for  structural  components, 
but  also  for  many  mechanical  and  electronic  components. 
However,  it  presents  the  other  approaches  available  for 
quantitative  treatment. 

3.  Quite  a  few  techniques  that  do  not  appear  in  government 
specifications,  but  which  industry  has  found  effective,  are 
presented. 

4.  Emphasis  is  placed  on  (a)  contract  management ,  and  (to) 
methods  to  design  for  required  reliability,  rather  than 
just  predict,  "control."  and  measure  it.  as  is  common  in 
other  courses. 

5.  Reliability  and  maintainability  are  treated  together  wher¬ 
ever  they  are  loyioally  managed,  designed,  or  analyzed 

together . 

(j  „  while  the  text  content  includes  more  "system  effectiveness" 
than  some  courses  by  that  name,  it  concentrates  on  just  the 
reliability  and  maintainability  contributions  to  system 
effectiveness,  to  avoid  dilution. 

7.  Cost- effectiveness  analysis  approaches,  to  determine  ccon- 
omically-achievable  reliability  and  maintainability,  are 
presented  in  some  detail. 

8.  Shipbuilding  and  ships  GFE  and  CFE  examples  are  used  wherever 
the  information  was  obtainable,  and  shipbuilding  critique 
obtained  on  all  text. 
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Although  the  great  majority  of  the  techniques  presented  are  well- 
established  and  proven,  some  are  still  controversial,  and  some 
are  simply  recommended  on  the  basis  of  industry  experience.  This 
is  to  be  expected  of  any  fast-developing  technology.  In  each 
case  the  text  words  will  usually  indicate  such  status. 

For  BuShips  top  management  courses,  Chapters  1,  2,  24,  25,  and 
26  are  used,  with  a  short  condensation  of  Chapters  3  through  23. 
for  middle  management  the  condensation  is  much  deeper.  For  the 
Technical  Codes  nearly  all  chapters  are  used  in  detail. 

In  order  to  achieve  the  above  objectives  several  approaches  have 
been  used.  Some  excellent  contributions  have  been  used  directly 
with  li.tT.1e  or  no  modification.  Much  material  of  significant 
content  has  been  rewritten  in  more  communicative  language.  About 
a  third  or  more  of  the  material  is  original  wirn  the  authors. 
BuShips  code  609.2  and  the  author  would  indeed  appreciate  re¬ 
ceiving  any  recommendations  for  improvement,  corrections ,  or 
criticisms  of  the  text.  It  will  have  to  be  updated  as  the 
technology  moves  ahead. 
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Chapter  CONTENTS 

1  INTRODUCTION  -  the  problem,  definitions,  course  coverage, 

CNO-CNM  dialogue,  BuShips  implementation 

2  REQUIREMENTS  DEFINITION  -  R&M,  planned  use,  R&D  plan,  PTA,  TDP 

3  SYSTEM  DEFINITION  -  approach,  models,  logic,  application 
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5  DATA  ACQUISITION  -  population,  problems,  estimating 

10  STATISTICAL  TECHNIQUES  -  sequential  analysis,  testing,  Monte 

Carlo,  design  of  experiments,  analysis  of  variance,  Boolean 

11  VERIFICATION  -  assurance,  applies  f  v<~  of  theory,  R  estimation, 

demonstration  testing,  sequential  sampling 

12  FAILURt  MuuES  &  EFFECTS  ANAL'ya  la  -  a^iicdth-n,  critical. 

items,  reducing  effects  of  failure 

13  DESIGN  FOR  R&M  -  oasic  reliable  design,  xel.iabil.ity  improve¬ 

ment  approaches,  maintainability  design 

14  HUMAN  FACTORS  -  n.an  as  an  element,  man-machine  interface, 

evaluation  of  nan  in  a  system 

15  DESIGN  REVIEW  -  phases,  checklists,  coverage,  effectiveness 

16  FAILURE  DIAGNOSIS  -  causes,  identi f i cat ion ,  corrective  action 

17  SPECIFICATIONS  -  R&M  consid  rations,  spec,  list,  abstracts 
13  PARTS  ENGINEERING  -  standar  s,  preferred  parts,  application, 

data,  specs,  stockroom,  find  ling,  traceability,  testing 

19  SUPPLIER  RELATIONSHIPS  -  qualification,  criteria,  specs, 

proposals,  surveys,  contract  negotiation,  evaluation,  control 

20  MANUFACTURE  &  OPERATION  -  suppliers,  manu faetur ing ,  test, 

delivery,  installation,  operation,  maintenance 

21  CONTRACTOR  ORGANIZATION  -  work  flew,  structure,  policy  & 

procedure,  responsibility  assignment,  education  &  manuals, 
technology  development,  change  control,  corrective  action 

22  TASK  DELINEATION  -  education,  design  to  R&M,  apportionment, 

prediction,  cost-effectiveness,  failure  mod  s ,  stref^s, 
strength,  human  factors,  design  review,  i  arts  control, 
reports,  corrective  action;  change,  supplier,  &  manufactur¬ 
ing  control;  failure  diagno  is,  data,  verification 

23  CONTRACT  PLANNING  -  requirements,  program  plans,  proposal 

management  <  evaluation  cost-effectiveness  p-  isions 

24  PROGRAM  CONT'  DL  -  R&D ,  shipbuilding,  responsibility,  contract 

negotiation,  contractor  evaluation  &  control,  surveillance 

25  SYSTEM  EFFECTIVENESS  -  application,  criteria,  factors , 

reliability,  reliability  &  maintainability,  models 

26  COST-EFFECTIVENESS  operational  experience,  cos  {.-effective¬ 
ness  balance,  tradeoff  analysis,  examples,  acquisition 
cost,  ownership  cost,  opportunities  for  improvement 

DEFINITIONS  -  reliability,  maintainability,  availability, 
effectiveness,  cost,  general 
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Cnapter  1 
INTRODUCE  ION 


To  talk  Reliability  to  the  Bureau  of  Ships  might  appear  to  be  a 
little  like  "bringing  coals  to  New  Castle".  The  endurance  of 
the  ships  you  have  designed  and  were  responsible  for  building  is 
traditional.  Ships  are  reliable,  they  respond  to  the  demand  when 
required  -  get  underway,  proceed  and  complete  an  assignment  and 
return.  The  country  gives  you  credit.  Innovations  in  shipbuilding 
you  have  taken  i,.  stride  ~  the  Nautilus,  the  George  Washington, 
the  Enterprise.  These  greats  are  testimonial  to  your  competency 
in  staying  abreast  of  the  new  technoioaius . 


1 .  STATEMENT  QF  TEE  FRO ELEM 

1.1  RELIABILITY  OF  EyUIPM ENT 

Then  why  talk  reliability?  The  reliability  we  will  be  talking 
about  \s  t-A'.-r  i e.iiabi.  Ly  of  equipment,  installed  in  your  ships, 
iou  lx  recognise  that,  some  of  the  equipment  furnished  by  the 
Bureau  and  carried  m  these  ships  does  cause  the  operating  forces 
problems.  Breakdowns  of  machinery  have  always  occurred.  It  was 
not,  in  the  old  days,  too  frequently  to  be  acceptable.  Today  the 
operating  forces  s?v  that  The  difficulty  of  maintenance  is  no 
3 on g e r  a e ceptable. 


The  Commander  In  Chiei-Paci fic  Fl oet  recently  said: 

"The  ever-increasing  complexity  of  shipboard  equipment 
continues  tc:  add  to  the  already  overextended  training 
requirements.  The  acceptance  of  shipboard  equipment 
which  exceeds  the  capabilities  e  Navy  personnel  to 
maintain  can  only  result  in  a  loss  of  fleet  readiness." 

The  Commander  In  Chi ef- Atlantic  Fleet  seconded  this: 

'Our  Fleet  is  becomir  •*  so  saturated  in  complexity  that  I 
ha,re  c  mortal  fear  we  may  bo  blindly  sailing  on  a  collision 
course  with  something  dread ful-- like  not  being  able  to  take 
the  Fleet  to  s~a  and  fight!" 

R.  Adm.  J.  O.  Cobb,  Asst.  Chief  of  Naval  Personnel,  (1)  stated: 

"The  baying  of  otaplex  systems  which  generate  more  and  more 
requirements  for  skilled  people  we  do  net  have,  leuds  to  an 
obvious  conclusion,  a  lot  of  hardware  goes  begging  for 
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1.2  RELIABILITY  VS.  COMPLEXITY 

You'll  recognize  ih.l  the  equipment  I'm  talking  about  is  largely 
(but  not  entirely;  electronics.  The  new  technologies  are  largely 


was  declassified  from  SECRET.  Twenty  five  years  ago  it  wasn't 
even  developed  adequately  to  place  aboard  ship.  In  the  last 
twenty- five  years  we  have  seen  electronics  on  board  ship  mush¬ 
room  from  a  small  shack  behind  the  pilot  bourse  to  spaces  filled 
with  consoles,  a  complex  array  of  antennas,  RADAR,  SONAR,  TACAN, 

LOR AN  and  so  forth.  Weaponry  and  defense  have  advanced  to  higher 
speed:  ,  greater  precision,  immensely  shortened  reaction  times. 

The  trend  of  the  advancing  technology  is  shown  in  Figure  1-4.  Of 
particular  interest  is  the  gradual  increase  of  installed  electric 
power  capacity.  Figure  1-5  shows  weapons  trends  for  the  same 
destroyers.  You  can  note  the  specialization  toward  Anti-Aircraft 
warfare  after  World  War  I,  then  toward  both  Anti-Aircraft  and  Anti- 
Submarine  warfare  after  World  war  II.  Looking  at  Figure  1-6,  we 
can  see  the  inspect  of  the  electronics  expansion  on  the  Destroyer 
Classes,  increases  in  weight ,  space,  power  required  and  erst.  The 
major  problem  the  operating  forces  find  is  a  decreasing  ability  to 
keep  the  exotic  new  systems  functioning. 

Concurrently,  there  has  l;,cn  a  trend  in  mechanical  equipment  toward 
higher  power,  greater  speed  of  rotation,  higher  temperatures  and 
pressures,  less  weight . 

This  trend  in  increased  capability  {more  complexity  in  electronics, 
greater  specific  performance  in  mechanical  areas)  has  taken  its  toll 
by  increasing  th"*  cost  and  difficulty  of  maintenance  and  reducing 
the  effectiveness  of  the  Navy,  This  is  why  Admiral  Schoeeh  said 
in  November  (7 ) . 

"System  Effectiveness,  and  its  fiscal  cor rollary  cost  effective¬ 
ness,  constitute  the  m  To.  important  single  concern  of  military 
RkD  Management.  ' 

i  ,  1 _ RELIABILITY  AS  A  MAN  AG  EM  I!  NT  PROBLEM 

Obviously,  we  can't  go  back  to  the  good  old  days .  The  new 
capabilities-  are  necessary.  Out  effort  must  be  to  meet  the 
challenge  by  learning  to  develop  systems  that  meet  the  present 
day  requirements  of  dependability,  while  staying  abreast  of  our 
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fleets  needs  in  specific  performance. 

In  an  attempt  to  reverse  the  trend  towards  higher  costs  and 
reduced  effectiveness — to  get  action  started  toward  a  major 
improvement  in  reliability  and  maintainability  of  Weapons  Systems 
--DOD  Instructions  3200.6  and  3200.9  have  been  issued.  These  and 
a  lot  of  other  letters  and  memos  have  been  written  and  speeches 
made  to  "resent  the  new  view  and  emphasize  the  n«ed.  The  effort 
has  centered  on  improving  the  management  of  development  contracts. 
Pressure  has  been  applied  and  will  be  applied  until  the  fleet 
can  live  with  the  equipment. 

These  new  ground  rules  for  R&D  management  have  been  compared  by 
Admiral  Booth  (4)  to  the  negotiation  of  a  contract  between  the 
CNO  and  the  Chief  of  Naval  Material. 

"The  Chief  of  Naval  Operations,  as  the  customer,  is  demanding 
a  materiel  program  which  wisely  invests  the  resource  made 
available  to  the  Navy." 

One  major  facet  of  the  new  rules  is  the  "economic  sanctions" 
applied.  Sound  planning  for  development  contracts,  including  the 
appropriate  consideration  of  reliability  and  maintainability,  is 
a  prerequisite  for  the  authorization  of  funding.  To  meet  the 
new  requirements  will,  for  the  Bureau,  mean  a  new  approach  to 
contracting.  But  first  we'll  discuss  the  old  words  in  thei- 
new  meaning. 

2 .  DEFINITIONS 

2.1  RELIABILITY  AND  MAINTAINABILITY 

The  words  Reliability  and  Maintainability  will  be  used  extensively, 
so  I  would  like  to  start  with  an  intuitive  definition.  Relia¬ 
bility  is  the  performance  characteristic  of  equipment  that  reflects 
its  ability  to  operate  satisfactorily  long  enough  to  complete  its 
assigned  mission.  It  is  an  index  of  the  excellence  of  the  design 
and  of  the  operational  integrity  of  a  product.  Higher  reliability 
means  fewer  breakdowns  -  longer  periods  of  trouble  free  operation. 
Maintainability  is  the  performance  characteristic  that  reflects, 
rapidity,  ease  and  economy  of  maintenance  and  repair.  Higher 
maintainability  means  reduced  requirements  for  skilled  personnel, 
less  down  time  for  equipment. 

The  engineering  approach  to  design  concerns  itself  with  specific 
functional  performance.  Will  the  equipment  do  what  we  want? 
Reliability  Engineering  asks  'How  long?"  Design  Engineering 
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starts  with  the  assumption  the  equipment  will  work.  Reliability 
engineering  starts  with  the  assumption  it  will  fail.  Design  for 
performance  is  concerned  with  how  "effective"  the  system  is  in 
operation.  Design  for  reliability  is  concerned  with  how  long 
the  system  can  function  without  failure. 

2.2  DESIGN  BASIS  OF  RELIABILITY 


How  are  systems  or  equipments  designed?  There  are  actually  very 
few  new  parts  in  any  design.  In  developing  a  n<_w  equipment  the 
designer  selects  parts  (bearings,  linkages,  seals,  power  supplies, 
servos)  that  have  been  previously  used,  adapting  them  t.o  his 
requirements.  Each  of  these  parts  has  some  history  of  appli¬ 
cations  and  some  history  of  failures.  Where  failures  occurred 
the  design  was  changed  until  a  satisfactory  design  was  achieved. 

As  a  result  of  experiment,  testing  and  trying  new  combinations, 
empirical  rules  for  the  use  of  the  part  have  been  developed.  The 
designer  of  a  new  system  uses  these  previously  developed  rules 
with  established  analytical  techniques  to  produce  a  usually 
acceptable  design. 

In  a  new  design,  the  designer  must  make  a  certain  number  of  trace- 
offs  or  compromises.  Usually  several  possible  configurations  are 
studied,  the  advantages  and  disadvantages  weighed  and  one  finally 
selected  which,  in  the  opinion  of  the  designer,  best  meets  his 
objectives.  Within  the  specific  performance  requirements,  two 
extremes  of  approach  might  be  found.  The  conservative,  or  over- 
design  approach,  emphasizes  high  reliability.  Excess  weight  or 
c  st,  even  marginal  performance,  may  be  accepted  to  assure 
reliable  performance.  (An  example  might  be  the  reduction  in 
hydraulic  working  pressure  on  the  periscope  hoisting  cylinders  to 
prevent  use  of  excess  pressure  on  the  seals).  The  optimistic,  or 
performance  oriented  approach,  emphasizes  development  of  specific 
performance  beyond  the  requirement,  or  low  weight  or  cost  (hoping 
the  reliability  will  be  adequate).  This  approach  represents  a 
Did  for  recognition,  or  the  solution  to  a  challenging  problem. 

Either  approach  can  lead  to  seri-'js  modifications  after  me  equip¬ 
ment  is  manufactured.  The  conservative  approach  may  incur  changes 
to  "fix"  overweight  or  poor  performance  problems.  The  optimistic 
approach  may  require  changes  to  "fix"  reliability  problems. 

But  Admiral  Schoech  said  in  November, 

"We  can  no  longer  afford  the  ’build  one  and  try  it’  approach 
with  a  subsequent  ’get  well’  effort  to  patch  on  reliability, 
maintainability,  etc." 
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The  intermediate  approach,  considering  all  the  requirements  for 
reliability  and  performance  in  each  decision  is  clearly  better 
than  either  extreme.  This  approach  requires  some  criteria, 
ground  rules  or  method  of  analysis  for  making  decisions.  To 
establish  such  criteria  or  methods  we  look  back  to  traditional 
design  metnods. 

Designers  design  from  experience.  They  use  knowledge  gained 
from  their  own  and  other  people's  experience  to  put  together 
an  equipment  that  will  work.  They  are  familiar  with  the  relation¬ 
ship  of  cause  and  effect.  In  their  experimentation  and  observing 
the  results  of  other  peoples  efforts  they  have  classified  some 
coni igur at  ions  as  "good"  -  they  work,  and  ethers  as  "poor"  -  they 
don't.  They  have  learned  that  properly  conducted  experiments 
are  repeatable  because  of  the  cause-effect  relationship. 

The  design  for  a  specified  reliability  can  take  the  same  approach. 
Failures  are  repeatable.  There  is  no  such  thine;  as  a  chance 
failure.  Every  failure  that  occurs  is  caused  by  the  implacable 
f'n-’rsH  nn  of  ohvsiral  1  aws  .  w*»ar.  nv'i''cfres?  and  progress  c 
deterioration  are  physical  events  caused  by  physical  conditions. 

A  part  used  in  a  system  is  subjected  to  the  operation  of  these 
laws  in  a  reasonably  consistent,  way.  It  may  be  expected  to 
survive  on  the  average  a  fairly  predictable  length  of  time. 

As  parts  were  originally  designed  and  developed,  their  reactions 
to  certain  combinations  of  pressures,  dimensions,  loadings, 
lubricants  were  evaluated.  Certain  combinations  were  found 
usually  successful,  other  combinations  were  rejected.  This  was 
the  source  of  the  analytical  design  rules  or  criteria  previously 
mentioned.  But  each  analytical  technique  uses  a  go-no  go,  good- 
hue  criteria.  We  can,  and  in  a  few  instances  have, determined, 
how  good  or  how  bad.  The  life  expectancy  for  each  part  under  a 
particular  set.  of  conditions,  ana  how  such  life  expectancy  varies 
with  changes  in  the  conditions,  can  be  established. 

A  system  composed  of  these  parts,  each  with  its  own  characteristic 
life,  exhibits  a  characteristic  "random"  failure  pattern.  Random, 
as  ur ed  here,  describes  a  situation  where  nearly  the  same  number 
of  failures  occur  in  any  two  equal  periods  of  time. 

Figure  1-10  portrays  a  system  of  several  parts,  each  failing 
(denoted  by  F)  at  its  characteristic  frequency.  The  system  failure 
frequency  is  shown  for  two  discrete  equal  periods  of  time. 

We  can  thus  establish  a  figure  of  merit,  charact  'fistic  of  the 
system,  tv)  evaluate  reliability.  One  such  figure  or  merit  we  call 
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the  Moan  Timo  Between  Failures  (MTBF) ,  computed  by  dividing  the 
total  ooeralina  time  bv  the  number  of  fan  t  o. 
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yirc  c  ‘  merit  can  be  used  to  evaluate  the  design.  It  can 
icted  in  the  design  stage,  before  production  of  equipment 
From  th'-  information  obtained  on  prior  test  programs  we 
ermine  w'  -ther  that  system  life  will  be  satisfactory,  as 
information  on  row  to  improve  it  by  selection  of  working 
levels,  tolerances  or  other  factors  having  an  effect  on 
e  of  the  parts.  This  is  a  significant  refinement  to  the 
lonal"  approach  to  design  in  that  part  life  at  the  design 
ons  is  predicted  and  compared  against  pre-ass igned  require- 
so  that  design  renditions  may  be  modified  to  improve  the 
characteristics  f  the  design  before  the  equipment  is 


k . 3  DESIGN  BASIS  OK  MAINTAIN  ABILITY 


We  have  talked  about  failures  as  if  this  were  the  end  of  the  line. 
It  i s  n ‘ t  so.  Failures  have  to  be  repaired.  They  may  be  repaired 
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the  task  can  be  expedited  by  changes  in  design.  Typical  examples 
are  improved  access  (particularly  emphasizing  short  life  items), 
modular  desiyn  or  planned  replacement  at  higher  levels  of  assembly 
to  reduce  detail  assembly  and  adjustment  times. 

As  we  have  indicated,  the  reliability  and  maintainability  achievable 
in  a  design  are  within  the  control  of  the  designer.  He  can  deter¬ 
mine  how  much  he  needs.  He  can  select  alternate  approaches,  each 
of  which  meets  his  prime  requirement.  He  can  select  the  one  which 
best  meets  his  secondary  objectives  (low  first  cost,  low  maintenance 
cost,  short  down  time).  Having  sel  cted  the  approach,  he  can 
design  the  equipment  so  that  the  predicted  failure  rat°s  of  the 
parts  will  not  cause  the  equipment  to  fail  more  frequently  than 
permissible,  or  so  that  the  estimated  repair  time  remains  within 
the  permissible  down  time. 

2.4  DEGRAD AT  ION  OF  RELIABILITY  AND  MAINTAINABILITY  IN  PRODUCTION 

AND  USE  "  ~  "  .  ~  .  . . 

The  designer  establishes  the  maximum  achievable  reliability  by  his 
design.  Poor  manufacturing  processes,  poor  inspection,  inadequate 
maintenance  or  improper  operation  can  reduce  the  observed  relia¬ 
bility  below  that  which  is  inherent  in  the  design.  The  development, 
of  reliability  in  a  design  can  be  compared  to  a  tree.  The  character¬ 
istics  of  the  tree  are  established  by  the  seed,  but  harsh  environ¬ 
ment  can  disfigure  or  dwarf  it.  The  design  is  the  "seed  of  the 
equipment .  The  equipment  can  never  be  better  than  the  design.  Put 
errors  on  the  production  line,  or  carelessness  m  maintenance  can 
prevent  the  inherent  reliability  from  being  achieved.  (Figure  1-13) 


The  approach  to  high  reliability  and  good  maintainability  is  sound 
engineering.  The  techniques  and  procedures  we  will  demonstrate  in 
this  course  are  those  techniques  and  procedures  a  'hr  'd'  designer 


S  fl  O  U  1  \~i  U  ii  C  in  designing.  The  various  program  aspects,  design 
practices  and  procurement  procedures  that  make  possible  the  achieve 
ment  of  reliability  in  design  and  its  retention  in  manufacture  md 
use  must  be  initiated,  controlled,  and  audited  by  the  eng i nee r 
responsible  for  the  procurement. 

3  .  COVERAGE  OF  RELIA  BI LITY  \  MAINTAINABILITY  TR  A  IN  INC.  C  PIT  St 
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This  coin  so  is  designed  to  provide  an  initial  < 
principles  of  design  and  procurement  for  high  \ 
maintainability.  In  the  short  time  planned,  v< 
the  engineer  to  the  principles  and  practices  a; 
the  tec)  ,  quos  used.  The  gradual  irerease  in 
the  methods  will  require  time,  practice,  and  s: 
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■farina  the  design  an  i  development .  The  re- 1  i  abi  lity  techniques  we 
wi  ’  1  be  loach  i.  rv :  re  prose  nt  improved  dec  ign  practices  ,  a  sharpening 
of  do.;  i  -n  b  i  s  c  ip]  1p.”s  ,  a  deliberate  orientation  of  the  design  to 
specific  r-'l  i  .toil  ity  requi  remonts  . 

In  the  development  of  new  systems,  Mr.  J.  W.  Rcvv-h,  Assistant 
Director  (Engineering  and  Management )  ODPRs E ,  do l'inos  (5)  the 
Reliability  and  Maintainability  policies  ' f  the  POD  as  fellows: 


1.  Reliability  and  Maintainability  uaals ,  stabed  in  quantit¬ 
ative,  miss  ion-responsive  br:;a  must  be  established. 

/.  The  Reliability  and  Mainta  mabi 1 ity  goals  shall  be  the 
basis  of  technically  realistic  roqu . rements  that  can  be  con¬ 
tractually  specified  with  appropriate  demonstration  plans . 


i .  Reliability  and  Maintainability  can  be  obtained  only  by 
sound  engineering  during  -lesion  and  development. 

4.  As  stated ,  Reliability  and  Maintainability  must  be  designed 
into  t  he  equipment  ,  but  must  be  designed  in  on  a  system  basis 
:nu  merit  be  subject  to  tra-i«.  -off  cons  i  derat  ions  with  al  1  other 
ei-  iieeterj:  1  i  es  each  a.>  weight  ,  sire,  cost,  etc. 

R*>]  i  abi  lit-/  m  I  Mai  tit  .unnbi  iity  are  the  responsibilities 
of  tin  •  *-o  ject  management  d  jpg  an  i  zat.  ion . 

v.  A'-aa vince  ot  Reliability  and  Maintainability  ro'h  irements 
»<  h  >  t".’em«-nts  can  be  oht  a  i  n**d  on  1  y  by  constant  monitoring  by 
’  h«  or...  jeot  manager  and  his  staff,  utilizing  carefully  con- 
et  •  i  i  plans  for  per  iodic  it'viev.-  and  f°r  selected  demonstrations. 

In  j  *  a  i-v  eye'le,  equipment  passes  three  stages ,  development, 
i  .  o(i ; .  i , and,  operation.  POD  Instructions  3200.6  and  3200.9, 
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But  the  problem  we  closer  ibed — the  high  cost  of  maintenance,  the 
dissatisfaction  of  the  operating  forces  —  concerns  equipment 
already  designed  and  built,  systems  in  the  Fleet  today.  In  thi 
equipment,  improvement  is  needed.  The  appro a  nh  fo  improvement 
this  equipment  must  be  the  same  as  that,  required  for  nc*  dev  el  • 
ment . 


Improved  reliability  and  improved  maintainability  can  only  be 
achieved  by  improving  the  ties  tun.  Requirements  must:  be  establ 
and  the  system  redos  i^ned  ir.d  rebuilt  as  necessary  to  meet  the 
requirements .  *  nc  sav  Lr.q  factor  i  s  that  a  large  part  of  the  vr. 

reliability  will  be  found  to  be  due  to  a  small  number  of  com per 
An  organized  search  for  the  bad  actors  with  improvement  in  mair 
tenance  of  operating  records  ana  reporting  of  failures  and  a 
systematic  analysis  of  the  total  system  each  time  a  part  of  it 
selected  for  change,  would  result  m  an  orderly,  economical  impr 
ment 

1 .  THE  CNO-CNM  DIALOGUE 

Mr.  Roach’s  first  point  in  DOD  policy  on  Reliability  and  Main! 
ability  was  that  qoals ,  stated  in  quantitative,  miss ion-respons 
terms  must  be  established.  In  a  speech  (J) ,  R.  Adm.  C.  T.  Boot' 
stated , 

"V.her.  we  state  an  operational  requirement,  we  3re  generating 
a  dialogue  between  the  CNO  and  the  Chiefs  of  Bureaus  which  !. 
the  basis  of  Ovir  mutual  understanding  of  the  product  we  expn 
and  the  cost  and  time  schedule  on  which  we  expect  it.  It  is 
the  course  of  this  dialogue  that  wc  must  learn  to  inject 
quantitative  reliability  criteria." 


I*,  is  in  this  dialogue  that  the  Bureau  must  work  with  the  operating 
tcrcos  to  do  fine  reliability  requirements ,  tempering  the  definition 
with  the  realities  of  -achievable  limits  as  well  as  cost  and  schedule. 
The  definition  should  include: 

(a)  The  level  of  essentiality  (or  importance  }  of  the  system, 

(b)  How  the  operating  forces  expect  to  use  the  system. 

(c)  How  frequently  and  hew  long  it  could  be  down  without 
materially  affecting  the  mission  or  operation. 

(d)  What  the  relation  of  this  system  is  to  other  installed 
systems . 

(e)  What  kind  of  skills  and  what  number  of  personnel  with 
these  skills  could  be  made  available  to  maintain  and  operate 
the  equipment.  What  else  the  personnel  are  required  to  main¬ 
tain  or  operate. 

(f)  When  the  equipment  is  needed  in  the  Fleet.  What  event, 
situation,  or  other  capability  defines  this  time. 

(g)  What  level  of  funding  is  planned  or  permissible  to 
acquire  and  support  the  system. 

It  is  important  to  note  that  the  CNO  cannot  set  realistic  require¬ 
ments  by  himself.  High  reliability  is  not  necessarily  the  goal; 
but  rather,  the  prime  objective  is  to  obtain  systems  that  will 
operate  satisfactorily  and  meet  the  mission  needs  at  a  reasonable 
cost  and  with  reasonable  time  schedules.  The  dialogue  .is  necessary 
to  establish  what  these  mission  needs  are  and  what  can  be  provided 
within  budget  and  time  constraints. 

d •  BUREAU  OF  SHIPS  IMPLEMENTATION 

5.1  HOW  RELIABILITY  IS  ACHIEVED 


Having  established  the  requirements,  the  next  stage,  implementa¬ 
tion,  is  the  Bureaus.  Definition  of  requir ements  and  their  speci¬ 
fication  in  numerical  terms  is  not  enough.  As  Mr.  Roach  pointed 
out,  reliability  must  be  designed  into  the  system.  It  must  be 
kep  ,  in  through  manufacturing,  use,  and  maintenance.  If  it's  not 
th  re  in  the  basic  design,  it  can't  be  put  there  except  by  fixing 
the  design.  And  changing  a  design,  and  retrofitting  the  equipment, 
if  far  more  expensive  than  doing  the  design  right  the  first  time  — 
before  the  equipment  is  produced. 
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The  practical  approach  to  achieving  the  r ecu i red  reliability  is 
controlling  the  designer,  requiring  him  to  give  adequate  consider¬ 
ation  to  reliability  and  maintainability  in  the  design.  This  is 
done  by  teaching  the  designer,  and  requiring  him  to  use  and  docu¬ 
ment  sound  engineering  disciplines,  practices  and  analyses.  Con¬ 
trol  of  his  activities  is  accomplished  by  formal  documented  audit 
of  his  considerations  and  decisions  by  we1 1-quali fied  designers 
in  his  field  --  usually  senior  designers  in  his  own  unit. 

Quantitative  requirements  are  necessary  to  describe  the  degree 
of  reliability  and  maintainability  desired.  Demonstration  is 
necessary  to  confirm  that  the  requirements  are  met,  but  the  veri¬ 
fication  must  be  supported  by  a  good,  solid  assurance  that  the 
designer  himself  is  considering  reliability  and  maintainability 
in  his  design  in  an  organized,  understanding,  and  effective  way. 
Later  sessions  will  explair  how  this  is  done  --  suffice  it  for 
the  present  to  say  this  can  be  done,  is  being  done  in  industry 
today. 

5.2  MANAGEMENT'S  TASK 

As  the  top  management  of  the  Bureau  of  Ships,  you  can  make  it 
possible  or  impossible  for  your  engineers  to  work  toward  improved 
reliability  and  maintainability.  We  won't  eliminate  your  problems. 
Management's  task  is  solving  problems.  In  the  solution  to  the 
problems  discussed  today  we  believe  that  the  approach  taken  by 
management  must  include: 

1.  Understanding  the  relationship  of  sound  engineering  to  true 
reliability  and  maintainability. 

2.  Understanding  the  relationship  of  reliability  and  maintain¬ 
ability  to  cost  of  acquisition  and  ownership. 

3.  Applying  this  understanding  to  the  management  of  the  Bureau's 
business  in  design,  development,  and  procurement. 

Paraphrasing  a  statement  of  Dr.  Harold  Biown  (6),  We  have  re¬ 
cently  surveyed  the  reliability  status  of  a  number  of  system 
development  programs  in  all  three  services.  Our  intention  was 
to  estimate  how  much  management  attention  is  being  given  to 
substantive  reliability  activities.  One  specific  action  that  I 
feel  needs  to  be  taken  without  delay  is  to  assure  that  those  with 
line  responsibility  for  development  management  at  all  levels  have 
sufficient  knowledge  of  reliabil  ’  iques  and  methodology 

to  perform  their  management  respons .  .ties  in  this  area. 


i 
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Let.  me  elaborate  a  minute  on  the  approach  to  developing  manage¬ 
ment  capability  in  improving  Reliability  and  Maintainability. 

First  an  understanding  of  the  concepts  taught.  'We  expect  to  con¬ 
vince  you  that  by  the  rigorous  control  of  the  contractors’  relia¬ 
bility  programs,  reliability  can  be  improved  in  the  design  and 
manufacture,  that  this  new  concept  is  not  only  practical,  and 
economical  in  the  new  development  programs,  but  will  also  provide 
you  with  increased  economy  and  less  time  ana  effort  lost  in  the 
main  part  of  your  business.  We  expect  to  convince  you  that,  the 
.reproaches  we  teach  are  good  engineering,  with  better  definition 
f  method  of  achieving  that  excellence  of  design  we  are  looking 
lor.  We  will  demonstrate  that  the  concepts  are  sound  and  the 
ccs t  reasonable. 

Second,  the  application  of  this  understanding  to  the  Bureaus 
bus iness : 

The  concept  of  design  for  a  specified  reliability  is  not  univer¬ 
sally  applied  across  the  Bureau  today  although  it  is  extensively 
used  in  electronic  areas  and  RkD  programs.  Integration  of  relia¬ 
bility  and  maintainability  requirements  on  a  total  shin  basis  is 
not  apparent  in  shipbuilding,  conversion  or  fleet  improvement, 
programs,  with  the  recognition  of  the  total  system  concept,  we 
expect  you  to  initiate  the  implementation  of  reliability  improve¬ 
ment  programs  as  a  part,  of  the  Fleet  Improvement  Program. 

The  effectiveness  of  a  reliability  improvement  program  depends 
not  only  on  the  available  control  teenniques  and  the  competence 
of  the  personnel  hut  also  on  management's  active  interest  and 
understanding  of  the  problems.  The  best  way  to  motivate  an  engin¬ 
eer  is  to  let  him  know  that  the  top  management  will  not  tolerate 
anything  less  than  his  hrul  efforts.  ?<.  evaluate  the  efforts  of 
the  eng’  u,.r  requires  that  the  top  management  read  and  understand 
the  progress  and  problem  reports.  To  pass  the  word  back  down 
requires  that  the  top  management  react  to  the  reports,  even  with 
as  little  as  a  hand-written  comment  or  request  for  more  informa¬ 
tion  . 

To  instill  in  the  engineers  the  concept  that  reliability  and 
maintainability  considerations  apply  universally,  rather  than 
only  in  RS<D,  one  obvious  step  is  to  develop  a  reporting  procedure 
that  gives  equal  emphasis  on  reliability  and  maintainability 
whatever  the  program.  The  PERT  and  milestone  concepts  are 
familiar  to  the  Bureau.  Whichever  concept  is  used,  a  standard 
reeuirement  that  each  report  include  a  section  on  reliability 
and  maintainability  goals,  achievements  and  problems  would  initiate 
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consideration  of  reliability  by  the  engineers  and  provide  manage¬ 
ment  with  visibility  of  the  level  of  consideration. 

The  establishment  of  Shipbuilding  and  Operating  Fleet  reliability 
improvement  programs  require  "oordination  across  the  Bureau  and 
with  the  CNO.  The  initiation  of  a  program  to  analyze  the  praaant 
situation  and  determine  what  the  present  problems  are,  to  establish 
reliability  and  maintainability  goals  for  each  system  in  which 
such  goals  are  applicable,  and  to  establish  reporting  systems  to 
provide  management  visibility  of  the  progress  toward  achieving 
the  goals,  .n  only  be  initiated  from  *-he  top  manag*  .nent  level. 

5.3  SUMMARY 

Why  should  you  work  toward  improvement  of  reliability  and  main¬ 
tainability?  I  can  summarize  in  a  few  words: 

1.  It  will  improve  the  effectiveness  of  the  Fleet. 

2.  It  offers  ultimate  dollar  savings  in  maintenance  repair,  and 
logistics  . 

3.  It  can  eliminate  the  need  and  reduce  *-he  cost  and  effort  of 
"fix"  programs. 

4.  It  wisely  invests  the  resources  made  available  by  the  customer 
by  matching  equipment  to  resources. 

5.  It  improves  customer  satisfaction. 

6.  But,  most  of  all,  cood  reliability  and  good  maintainability 
are  "good''  design,  achieved  by  the  logical  application  of 
sound  engineering  analytical  methods. 

As  top  managers  of  the  engineering  effort,  you  have  a  heritage  t© 
be  proud  of,  the  outstanding  capability  and  performance  of  th« 
ships  you  designed  and  built  in  the  past.  As  the  technical  co*p© 
of  the  Navy,  it  is  up  to  you  to  assume  the  mantle  of  leadership 
to  maintain  in  this  complex  technological  era  that  traditional 
excellence  of  *  sign  to  assure  t.iat  the  equipment  furnished  to 
the  Fleet  reflects  "good"  engineering  and  lependable  performance. 
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Chapter  2 

REQUIREMENTS  DEFINITION 

The  development  of  reliability  requirements  requires  an  under¬ 
standing  of  what  reliability  is  and  how  it  is  achieved. 

1  *  DEFINITIONS  _0 F  RELIABILITY  AND  MAINTAINABILITY 

Reliability  and  maintainability  are  performance  characteristics 
cf  systems.  They  express  how  well  functional  performance  capa¬ 
bility  is  kept  available. 

1.1  RELIABILITY 

Reliability  is  defined  as  "the  probability  that  systems  or  com¬ 
ponents  will  perform  their  intended  function  for  a  specified 
period  under  stated  conait ions . "  Probability  means  the  fraction 
of  attempted  uses  of  the  system  that  will  be  successful.  A 
parameter  of  interest  in  the  measure  of  reliability  is  the  Mean 
Time  Between  Failures,  defined  as  the  average  Stress  Time  be¬ 
tween  failures.  Reliability  is  defined  by  tnreo  factors: 

{ a )  The  intended  us^  or  function  regu  i:  'd  to  be  performed . 

From  the  intended  use  we  derive  a  definition  of  failures 
the  incapability  of  performing  the  function. 

(b)  The  specified  period.  From  the  intended  use,  we  can 
determine  how  many  periods  or  cycles  of  use  will  occur 
on  the  average  for  each  expected  tai  lure.  For  an  equip¬ 
ment  with  an  MTBF  of  240  hours,  our  reliability  for  a 

2 4 -hour  period  would  be  90%.  That  is,  about  one  day  in 
ten  we  should  not  be  surprised  to  have  it  down. 

(c)  The  stated  conditions.  For  any  system,  a  severe  environ 
mont  will  reduce  the  reliability,  increase  the  average 
frequency  of  failures.  Environment  includes  weather , 
imposed  stresses  such  as  temperature  or  vibration  and 
the  human  '‘climate",  skills  of  operators. 

1.2  MAINTAINABILITY 

Maintainability  is  defined  as  the  speed  or  economy  with  which  a 
system  or  component  can  be  kept  in,  and/or  restored  to  full  per¬ 
formance  capability. 


A  maintainability  f"nct  i  n  i  s  used  to  quantify  mauitai  nab  i 1 i ly . 

It  is  defined  as  "the  pro.nahi  1  i  ty  that  when  maintenance  action 
is  initiated  under  stated  onditions ,  a  failed  item  will  be  re¬ 
stored  to  operable  conditions  within  a  total  specified  downtime . " 
Again  it  is  defined  by  three  factors: 

(a)  Definitions  of  failure.  This  is  the  same  as  for  relia¬ 
bility. 

(b)  The  specified  period  of  time.  This  is  the  tinu  between 
occurrence  of  a  failure  and  restoration  to  performance 
of  the  function  that  can  be  tolerated  within  the  planned 
use  of  the  equipment.  For  a  system,  the-  sum  of  ail  re¬ 
storation  times  divided  by  the  number  of  failures  is 
called  the  Mean  Time  to  Restore  (MTTR ) . 

(c)  The  stated  conditions.  Conditions  under  which  a  repair 
or  restoration  action  occur  include  the  numbers  and 
skills  of  personnel,  the  restoration  philosophy,  logis¬ 
tic  support  (tools,  equipment,  parts),  instructions, 
the  working  environment. 

2  .  TJIK_  INTFNTr.D  UFE 

2  .  I  _ Tiff:  FUNCTION 

The  purpose  of  erect it»y  a  system  is  to  provide  for  the  perform¬ 
ance  of  a  function,  such  as  detection  or  tracking  of  targets, 
steering  a  ship,  or  whatever .  The  user  is  interested  in  the 
performance  of  that  function  at  specified  times  based  on  the 
nature  of  his  missions.  The  penalty  for  failure  •-  f  the  function 
can  be  assessed  from  the  nature  of  the  mission. 

Development  of  a  technical  approach  presupposes  the  recognition 
of  a  requirement  and  of  a  capability.  The  requ  i  reiaent  must  be 
evaluated  in  terms  of  the  operational  tunct i on •  The  capability 
is  defined  in  terms  of  a  technique  or  techno- logy  related  to  the 
operational  funct ion .  This  capability  includes  the  ability  to 
continue  performing  the  funct ion  with  failures  at  an  acceptable 
rate  and  within  the  capabi 1 i t i es  of  the  repa  t  r  forces  to  main¬ 
tain. 

2.2  THF  CAPABILITY 

The  technique  or  technology  proposed  w ill  include  the  basic  nature 
of  the  systems,  radar,  computers ,  or  hydraulics.  The  function 


to  be  performed  v- ill  indicate  h  w  the  systems  will  be  used,  duty 
cycle,  and  probable  environment.  The  penalties  associated  with 
equipment  failure  and  failure  t  •  repair  with  ,  a  specified  time 
can  be  predicted  and  values  of  reliability  expected  to  be  accep¬ 
table  proposed.  These  values  can  be  compared  against  the  present 
capability  of  whatever  industry  is  involved  as  shown  by  presently 
available  similar  systems  and  an  estimate  made  of  the  amount  of 
development  effort  which  will  be  required  tc  achieve  the  accep¬ 
table  value. 

2.3  THE  REQUIR EM ENT 

Every  system  developed  must  be  responsive  to  a  General  Operating 
Requirement.  The  need  for  the  system  and  the1  nature  of  it?  use 
must  be  stated  in  or  inferred  from  such  a  requirement.  .3  system 
to  be  used  in  a  ship  must  be  responsive  to  the  functions  imposed 
by  the  missions  and  tasks  assigned  to  that  ship. 

In  attempting  to  define  a  requirement  for  reliability,  these 
questions  must  be  asked  and  answered; 

(a)  What  is  meant  by  the  failure  of  the  system?  The  answer 
can  be  stated  in  simple  terms:  the  failure  to  per  form 
its  function.  This  requires  the  statement  of  function 
in  specific  quantitative  terms  with  specified  tolerances . 
For  example,  a  radio  transmitter  may  send  out  a  signal. 

I f  the  signal  is  too  weak  the  receiver  will  not  receive 
it.  This  must  be  considered  a  system  failure.  So  func¬ 
tion  cannot  be  defined  as  transmitting  the  s ignal  but 
must  include  a  range  of  power  transmission  as  we  1 1 . 
Carrier  frequency,  noise,  directivity  and  other  factors 
■st  be  included  in  the  definition.  (dee  eh  apt  er  )  . 


How  frequent  lv  mav  failures  be  tolerated?  The  do  term,  in- 
at  ion  of  this  answer  must  be  made  hy  the  intended  user. 
Any  reliability  desired  can  be  developed  into  a  system , 
but  any  incremental  increase  must  be  developed  at  the 
expense  of  cost,  time,  weight.  Or  capacity.  The  ti  ado-off 
m  ust  b  v  m  a  d  e  ,  start,  in  g  it  urn  a  n  o  r  i  g  i  n  a  i  ost  i  m  a  t  e  o  f 
capability  versus  cost,  by  com par i r  i  the  penalties  assoc¬ 
iated  with  various  levels  of  development  -f  reliability 
until  a  satisfactory  solution  is  found. 


(c)  what  will  be  the  environmental  conditions _ surr  Minding 

the  use  and  oop lovment  of  the  svstem.’  This  question 
refers  tc  natural  and  induced  envit  aments  such  us  expos¬ 
ure  to  -weather,  high  ar~,'k'at  ions ,  sudden  shocks.  It. 


a  1 1> o  refers  to  the  "human  r  l  j.mate"  ,  skills  of  operators 
and  maintenance  personnel.  It  concerns  natural  habi 

tat  of  the  system  during  its  lifetime  including  periods 
of  inactivity  as  well  as  activity.  The  environment  v  1 
exert  a  large  measure  of  control  on  the  reliability 
achievable  within  present  capabilities  as  well  as  cost 
o  f  tin pr o v eraent . 

(a)  Wh at  is  the  planned  cycle  of  operations?  This  question 
refers  to  the  frequency  as  well  as  the  duration  ot  oper- 
af  ions.  It  describes  the  functions  to  be  performed  on 
each  type  of  mission,  the  length  of  time  the  functions 
are  reeded,  covering  each  and  every  mode  of  operation 
of  the  system. 

These  quest  ions  are  answered  in  the  objective  of  the  development 
Let's  look  at  the  Research  and  Development  Plan, 


3  .  Tilt;  R  ESI' ARCH  ANT  D  FT  DIP  PM  ENT  PLAN 

i_._l _ MISSION  ORIENTATION 

The  Research  and  Development  P roar  am  (3)  (except  for  basic  re¬ 
search)  is  orient  ed  toward  specific  miss  ions  m  particular  en¬ 
vironments.  The  bas  ic  input  into  the  R&D  program  comes  from  th<» 
Navy  and  Joint.  Lone  Range  strategic  Studies,  which  define  the 
future  roles  and  mission  of  the  Navy.  The  ' ong  range  threat, 
potential  capabilities  of  possible  enemies  and  the  expected 
political  climate  are  assessed .  Where  the  long  range  studies 
assess  the  period  beyond  ten  years,  the  Saw  Mid-Range  Study  is 
concerned  'ith  the  period  out  to  ten  year.*:  .  he  Joint  Strategic 
Object i vry  Plan  providi  objectives  for  tht  5-8  year  period. 

Upon  issue  of  a.»  edit  i;  n  of  the  Navy’s  proposed  mid-range  ship¬ 
building  objectives ,  operational  commands  having  cogni zant  in¬ 
ter  at  cuijrni  t  recommendations  on  missions  and  tasks  to  the 
DC N C  { F 1  e e t  Op e r  a t  i o ns  R e a d  i  n e ss  )  . 

The  statement,  of  Mission  and  Tasks  approved  by  the  CNO  for  each 
type  of  U.  S.  Naval  ship  p-owixs  the  key  to  a  ship’s  ultimate 
capabi  1  it ies  ,  charact  er  ist  res  and  cost.  it  turn  isb  •'s  a  broad 
statement  of  the  purpose  tor  which  the  ship  is  to  be  designed 
and  the  tasks  which  the  ship  ran  be  expected  to  accomplish. 

The  sponsor  for  the  type  ship  in  OPNAV  amplifies  the  informa¬ 
tion  contained  in  the  statement  of  missions  and  tasks  into  a 
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single  Page  characteristics  delineating  the  significant  features 
and  capabilities  of  the  new  ship  which  is  furnished  to  the  chair¬ 
man  of  the  Chips  Characteristic  Board.  The  type  sponsor  prepares 
formal  ;*D  requirements  to  provide  capabilities  required  but  not 
yet  developed.  The  5-yec.r  Force  Structure  and  Financial  Program 
initiate  the  start  of  budgetary  action  for  the  acquisitions  of 
the  hardware. 

The  development  and  establishment  of  the  Five  Year  Force  Struc¬ 
ture  and  Financial  Program  (FYFS&FP)  have  emphasized  the  necessity 
for  defining  the  Navy’s  mid-range  shipbuilding  and  conversion 
program  w'ith  an  accuracy'  and  in  detail  comparable  to  the  budget 
submission  increment.  The  shipbuilding  and  conversion  programs 
submitted  to  the  Secretar"  of  Defense  for  approval  must  there¬ 
fore  be  justified  by  the  Navy  in  terms  of  requirements,-  technol¬ 
ogical  feasibility,  production  availability,  characteristics  and 
cost . 

? ,2  DEVELOPMENT  OF  SHIPS 


The  Ships  Characteristic  Board  has  one  prime  objective,  (Figure 
2-7),  to  insure  through  timely  recommend -.cions  to  the  Chief  cf 
Naval  Operations  that  the  characteristics  of  all  naval  vessels 
not  only  meet,  but  anticipate  wherever  possible,  the  require¬ 
ments  of  naval  warfare  incident  to  approved  mission  and  tasks. 

The  specific  fiiks  performed  by  the  SCB  are: 

(a)  With  regard  to  all  naval  vessels: 

(1)  to  recommend,  based  upon  primary  guidance  from  the 
Standing  Committee,  Shipbuilding  and  Conversion, 
tdie  nature  and  extent  of  .  ich  installations  as  may 
be  necessary  to  meet  operational  requirements  after 
consider:.!  of  their  effect  upon  other  character¬ 
ise!' u  and  when  applicable  the  installation  of  items 
still  in  a  research  and  development  status,  after 
consideration  of  their  compatibility  with  research 
and  development  plans. 

(2)  To  t eviev  the  arrangement  of  material,  instruments, 
and  facilities  to  ensure  efficiency  in  operational 
use . 

(3)  To  make  recommendation  to  CNO  and  the  developmental 
agencies  relative  to  the  adequacy,  weight  and  moment 
requirements,  compatibility,  etc.,  on  all  types  of 
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developments  of  shipboard  equipment,  particularly 
electronic  equipment,  in  order  that  new  equipment 
shall  be  adaptable  to  shipboard  utilization  in  ful¬ 
filling  operational  requirements. 

(b)  With  regard  to  new  construction,  conversion,  and  modern¬ 
ization  of  ships,  landing  and  service  craft,  and  con¬ 
sideration  of  merchant  type  ships  planned  for  Naval 
acquisitions 

(1)  To  study  the  requirements  and  guidance  furnished 
and  from  such  study  develop  the  broad  ship  charac¬ 
teristics  which  will  support  the  mission  and  tasks 
ass igned . 

(2)  On  the  basis  of  estimates  furnished  by  the  Material 
Bureaus  in  connection  with  design  s Ladies,  to  advise 
the  Deputy  Chief  of  Naval  Operations  (FO&R)  and  the 
Chairman,  Standing  Committee  on  Shipbuilding  and 
Conversion,  of  the  probable  costs  of  the  ships  in 
the  program. 

(3)  To  recommend  the  characteristics  in  such  detail  as 
necessary  to  guide  the  bureaus  in  their  preparation 
of  plans  and  specifications. 

(4)  To  review  pre-characteristics  design  studies  before 
the  annual  program  is  developed  by  the  Standing 
Committee  on  Shipbuilding  and  Conversion,  and  before 
detailed  plans  and  specifications  are  finalized  by 
BuShips ,  and  recommend  changes  when  required. 

The  characteristics  of  the  ship  are  generally  defined  in  terms 
of  speed,  cruising  radius,  type  of  propulsion,  size,  weapons, 
and  other  special  equipment  to  support  the  missions  and  tasks. 

In  the  effort  to  anticipate  the  needs  of  the  Fleet  and  to  match 
or  anticipate  potential  enemy  capabilities,  the  early  introduc¬ 
tion  of  newly  developed  capabilities  into  the  fleet  is  mandatory. 

Ti.ese  new  capabilities  are  being  developed  today  (Figure  2-9) 
and  are  planned  for  future  development  in  the  Naval  Material 
Research  Objectives.  Each  system  or  capability  in  a  ship  had  at 
one  time  a  development  phase.  (Figure  2-10). 

3.3  DEVELOPMENT  OF  SYSTEMS 

Several  years  ago  it  was  realized  that  significant  improvements 
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were  required  in  RS<D  management  to  avoid  the  large  cost  overruns, 
schedule  slippages  and  performance/design  changes  that  had  become 
a  pattern  in  major  development  projects.  During  these  past 
several  years,  in  order  to  effect  significant  improvements  the 
DOD  has  tried  a  number  of  relatively  new  things  (not  necessarily 
new  in  concept  but  in  emphasis)  among  which  are  Incentive  Con¬ 
tracting.  Contractor  Performance  Evaluation.  fo^t-Ef fectiveness 
Analysis,  Categorization  of  R&D  and  Project  Definition  Phase. 

The  most  important  objective  of  the  PDP  is  to  provide  an  adequate 
basis  to  assure  that  management  decisions  to  proceed  with,  cancel 
or  change  development  projects  are  made  on  a  total  system  and 
total  cost  basis  which  includes  realistic  cost  and  schedule 
estimates  for  the  production  phase. 

The  other  objectives  are  to: 

(a)  Establish  firm  and  realistic  specifications. 

(b)  Define  precisely  interfaces  and  responsibilities. 

(c)  Identify  high  risk  areas. 

(d)  Validate  technical  approaches. 

(e)  Establish  firm  and  realistic  schedules  and  cost 
estimates  for  the  production  phase. 

(f)  Establish  schedules  and  cost  estimates  for  planning 
purposes  for  the  total  project  (including  production, 
operation  and  maintenance) . 

PDP  can  be  considered  to  be  one  step  in  a  series  of  steps  in  the 
research  to  production  sequence.  It  is  that  step  which  immediately 
precedes  the  full  scale  development  and  is  the  means  of  defining 
it.  The  steps  prior  to  PDF  are  necessary  to  assure  that  the 
proposed  development  project  is  ready  for  PDP.  These  include 
technology  and  building  block  component  developments  which  are 
accomplished  without  specific  reference  to  the  proposed  system 
development  program,  and  studies  specifically  aimed  at  the  pro¬ 
posed  development,  such  as  trade-off  studies,  feasibility  studies, 
cost-effectiveness,  operations  research,  etc.  The  development 
and  studies  prior  to  PDP  must  assure  that  the  prerequisites  to 
PDP  have  been  met: 

On  January  18,  1963,  Dr.  Brown  sent  a  memorandum  to  the  Depart¬ 
mental  Assistant  Secretaries  (R&D)  covering  several  major  concepts 
in  the  management  of  research  and  engineering.  One  section  of 
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this  memorandum  treated  Project  Definition  and  set  forth  positive 
ground  rules  for  its  application  to  new  projects.  These  ground 
rules  required  that  PDP  be  used  for  all  new  Engineering  Develop¬ 
ment  and  Operational  System  Development  projects  with  cunulative 
RDT&E  funds  of  $25  million  or  more,  and  provided  for  application 
of  PDP  to  other  projects  at  the  direction  of  DDR&E  or  the  option 
of  the  department.  DoD  Directive  3200.9  -,hich  was  just  issued, 
includes  these  same  ground  rules  for  application  with  one  addition: 
that  Engineering  Development  or  Operational  System  Development 
projects  with  anticipated  expenditures  for  production  investment 
of  $100  million  or  more  are  also  required  to  use  a  PDP. 

The  terms  Engineering  Development  and  Operational  System  Develop¬ 
ment  and  their  place  in  the  R&D  structure  are  outlined  in  DOD 
Instruction  3200.6.  The  Engineering  Development  and  Operational 
System  Development  categories  are  the  last  development  categories 
in  the  research  to  production  sequence  and  are  developments 
intended  for  Service  use.  Inadequate  or  tardy  definition  of 
these  projects  results  in  drastic  consequences  in  terms  of  total 
costs  (including  R&D,  production,  operation  and  maintenance), 
schedules  and  operational  effectiveness. 

The  Project  Definition  Phase  (Phase  I)  is  a  formal  step  preceding 
full  scale  development  (Phase  II)  during  which  preliminary  engin¬ 
eering,  and  contract  and  management  planning  are  accomplished  in 
an  environment  that  encourages  realism  and  objectivity. 

While  the  project  definition  phase  requirement  app’ies  to  new 
projects  of  $25,000,000  or  more,  the  basic  concepts  of  manage¬ 
ment  of  a  development  are  applicable  across  the  board.  Where 
the  basic  concepts  were  not  followed  in  the  original  development 
of  equipments  in  use  in  the  fleet  today,  we  still  have  the  design 
and  SOFIX  problems.  And  they  won't  go  away.  As  each  problem 
is  identified,  the  fire  drill  starts  again. 

3.4  NEW  DEVELOPMENT:  A  MANAGEMENT  PROBLEM 

The  problems  to  which  DOD  Instructions  3200.6  and  3200.9  are 
addressed  are  not  technological  problems,  but  problems  of  manage¬ 
ment.  Let's  look  at  the  structure  of  the  dialogue  between  the 
CNO  and  the  NMSE  (Figure  2-13). 

The  various  documents  covering  the  definition  of  requirements  are: 

(a)  General  Operational  Requirement  (GOR) .  A  GOR  is  a  gen¬ 
eralized  statement  of  needed  operational  capability  pre¬ 
pared  by  the  CNO. 
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(b)  Tentative  Specific  Op<_rat.  i  onal  Rogu  i  r  omjsn  *  (THOR  i  .  Trio 
TSOR  is  a  document  -r  i  a  i  nuted  ’ey  the  CNO  hv  wine1,  the 
CNO  requests  cert  a  in  informal  ton  -a  f  i  technical  nature 
which  is  necessary  in  or- ier  to  determine  if  a  valid  Navy 
research  and  development  requirement  exists. 

(c)  Proposed  Technical  Approaches  (I*TA):  The  1 are  docu¬ 
ments  prepared  by  the  NM  'L  for  the  CNO  out lining  tech¬ 
nical  approaches  by  which  a  particular  capabilitv  m  iy 
be  achieved. 

(d)  Speci  f  ic  Operat  jona  l  R> i  remont  (SOR)  :  The  rfOR  is  i 
document  by  which  the-  CNO  states  tie-  rved  for  the  dve  1  - 
oprnent  of  a  particular  operat ional  capability. 

(c-)  Advanced  Deielopmenc  Objective  (APO)  .  An  ADO  is  .1  ciocu- 
ment  prepared  by  the  CNO  and  addressed  to  t no  CNM  wn i ch 
states  a  need  tc  conduct  certain  experimental  stud  its, 
tests  and  development  effort  for  tlv.  purp  se  of  estab¬ 
lishing  the  potential  capabilities  of  a  new  weapon  con¬ 
cept  ,  the  technological  feasibility  of  develop;;'.,;  .;i  new 
system,  and  to  develop  creator  accuracy  in  t ho  cost , 
time,  and  performance  estimates  required  t  establish 
financial  acceptability  of  a  new  system. 


(f)  Technical  Development 
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programs  be  continuously  appraised  and  reviewed  in  order  tc.  per¬ 
mit  timely  reallocation  of  resource-  -ar  procram  curt,  a  i  lm-t  nt  when 
ever  such  action  appears  to  be  rc  ,u  1  red .  In  ardor  to  provide 
for  this  appraisal,  standard;  zed  reporting  procedures  haw  been 
established.  The  following  paragraphs  describe  several  of  these 
management -or iented  report s . 

(a)  Project  Report.  DD  Form  A 1  -  contains  the  basic  program 
information  required  by  management  for  the  an.ilss  is  and 
review’  of  RCTf.  E  proiects  in  the  POP  Research  arid  Kxpl  -r- 
atory  Development  r ategcrios. 
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(b)  Monthly  Project  Evaluation  (MPE) .  The  MPE  is  a  monthly 
report  submitted  to  the  CNO  by  the  bureau  cr  office  having 
management  responsibility  for  a  project  in  advanced  de¬ 
velopment,  engineering  development,  or  operational  systems 
development.  The  Purpose  of  the  MPE  is  to  direct  the 
attention  of  the  top  Navy  RDT&E  management  echelon  tc 
present  or  potential  problem  areas  in  the  RDT&E  program.. 

( c )  Research  and  Exploratory  Development  Program  Highlights. 
This  report  is  tc  keep  RDT&E  administrators  and  managers 
informed  as  to  progress,  or  lack  thereof,  *  -wards  objec¬ 
tives  within  the  categories  of  Research  ana  Exploratory 
Developments .  These  highlights  include  all  significant 
accomplishments  and  problems,  actual  or  anticipated, 
with  in  the  approved  programs  for  Research  and  Exploratory 
Development.  Program  highlights  are  reported  on  an 
exception  basis. 

(d)  Hotline  Report.  This  report  provides  a  formal  method  of 
ensuring  that  the  ASN(R&D)  and  DCNO(D)  are  made  quickly 
aware  of  RDT&E  problems  which  are,  or  have  the  potential 
for,  seriously  affecting  RDT&E  projects.  This  report 
will  provide  interim  coverage  when  major  or  critical 
problems  or  other  significant  events  occur  or  are  anti¬ 
cipated  between  reqular  monthly  progress  reports. 

(e)  Quarterly  Project  Reliability  Summary.  This  report  serves 
as 

(1)  A  reliability  annex  to  the  TOP  summary  by  providing 
the  minimum  acceptable  reliability  requirements  and 
the  contract  goals  as  the  basis  of  the  reliability 
rating  (in  the  Monthly  Project  Evaluation)  of  each 
project  in  engineering  development  and  operational 
systems  development,  and 

(2)  A  convenient  quarterly  progress  report  to  top>  Navy 
Research  and  Development  management  in  these  two 
categories  of  systems  development. 

3.5  IMPACT  OF  PROPOSED  TECHNICAL  APPROACHES 


The  Chief  of  Naval  Operations  has  been  charged  with  the  respon¬ 
sibility  of  developing  the  maximum  capabilities  in  the  fleet 
consistent  with  the  strategic  objectives  and  r he  Five-Year  Force 
Structure  and  Financial  Plan.  In  the  evaluation  of  needs  to 
support  Missions  and  Task  against  Capabilities,  Schedules  and 
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Costs,  the  Proposed  Technical  Approaches  constitute  the  Bureau ' s 
bid  on  the  job  offered  by  a  GOR  or  TSOR.  The  CNO,  in  his  nanuae- 
ment ,  must  try  to  spend  his  money  in  the  way  most  likely  te¬ 
ach  i  eve  the  mrst  pressing  of  his  needs.  The  PTA  must  sell  the 
Bureau's  understanding  of  the  need  ana  appreciat ion  of  the  sound¬ 
ness  of  the  approach.  Failure  to  convince  the  CNO  that  the 
equipment  can  be  provided,  within  the  required  time,  within  bud¬ 
get  limitations  and  with  adequate  capability,  may  cause  him.  *o 
decide  that  to"  risk  is  too  great  t-1  pursue  the  pr>.  iect .  An  SOF 
might  never  be  issued. 

4  ’  CONTENT!  OF  A  PROPOSED  TECHN ICAL  APPRO  AC:  • 

Reference  (6 '  requires  that  the  PTA  contain,  in  aid  it  ion  to  the 
functional  and  operational  description  of  the  system  and  the 
problem  to  be  solved,  an  estimate  of  the  operational  of  fert iveness 
of  the  proposed  system.  This  shall  be  stated  m  terms  of  per¬ 
formance  reliability,  operability  and  maintainahi  \  ty.  Alterna¬ 
tives  ?,n  performance,  cost  and  development  time  arc  required  t 
bracket  the  proposal  in  performance  and  dar  t ion  of  the  develop¬ 
ment  schedule. 

4.1  RELIABILITY  .ANT)  MAINTAINABILITY 

4.1.1  The  development  of  the  reliability  excect  tram  each  r 
the  alternatives  should  consider  the  present  st  at  e  or  capab  i  .1  i  tv 
of  the  particular  industry  as  wo  1 1  as  the  ant icip'utod  difficulty 
in  improvement.  The  cost  and  time  to  -level os'  the  equipment  wish 
present  state  of  the  art  reliability  and  the-  cost  an-;,  t  ire  t-  oto 
to  develop  the  maximum  feasible  reliability  within  the  schedu !  <. 
constraints  should  both  be  shown.  Both  values  should  be  com:  cod 
to  the  assumed  acceptable  value. 

4.1.2  Similarly  the  development  o  t  present  indu  try  -  >;\w 
and  maximum  maintainability  development  within  t  he  sole-  r:  !< 
straints  should  be  evaluated  and  shewn  in  the  altern.it  ives 

the  PTA. 

4.1.3  The  cost  consequences  for  each  level  ot  r-.-liabilif 
maintainabi  1  ity  should  be  est  imat  ed  us  ing  the  planne  :  „  per  it  i 
duty  cycles  previously  discussed  and  estimated  number  f  tai 
and  consequent  cost  of  repairs  as  <  11  as  int  ;  ci  p.ite  •  c.  .  t.  t 
maintenance  and  operation  (Figure  2  —  1“*  > .  The  impact  e:i  p.-rs- 
numbers  required  and  training  requirements  shod.:  he  est  ;*r.  *?.  -. 
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4.2  DEVELOPMENT  OF  RELIABILITY  REQUIREMENTS 

An  example  of  the  development  of  reliability  and  maintainability 
requirements  is  provided. 

4.2.1  We  consider  a  requirement  for  a  shipboard  fire  control 
system.  The  function  is  tc  actively  defend  the  ship  against 
enemy  aircraft.  Since  the  system  is  quite  complex,,  we  break  it 
down  ' nto  its  subsystems.  The  four  subsystems  are: 

(a)  Detection  system:  to  detect  aircraft  approaching  the 
ship. 

(b)  Tracking  system;  to  provide  continuously  the  slant  range 
and  bearing  of  the  designated  targets. 

(c)  The  fire  control  direc  or :  consisting  of  director  radar, 
computer  and  controls . 

(d)  The  weapon;  a  surface  to  air  missile. 

The  weapon  and  director  system  are  already  developed  and  avail¬ 
able.  The  system  to  be  developed  is  the  detection  and  tracking 
system.  Acquisition  range  required  is  40  to  50  miles  based  on 
assumed  aircraft  speed,  time  required  to  develop  tracking  inform¬ 
ation  and  reaction  time  for  target  acquisition  and  time  of  flight 
of  missile.  The  a  oach  selected  is  a  single  radar  with  search 
and  tracking  capabilities  with  a  computer  to  convert  range  and 
bearing  to  predicted  position. 

From  comparison  of  data  on  operational  systems  very  similar  to 
the  proposed  new  designs  v_  can  establish  estimates  for  certain 
parameters  of  the  new  systems.  These  are  considered  the  para¬ 
meters  achievable  with  present  design  methods.  Performance  is 
defined  as  the  probability  that  the  system,  when  operating  with¬ 
in  specification  v.’i.Ll  accomplish  its  function.  For  example, 
performance  for  the  search  r  idar  is  the  fraction  of  the  time 
that  approaching  aircraft  will  be  detected  before  they  reach  the 
minimum  acceptable  acquisition  range  of  40  miles.  For  the 
tracking  mode  it  is  the  fraction  of  detected  aircraft  successfully 
identified  to  the  fire  control  director. 

Search  Mode  Tracking  Mode 

Performance  .95  .99 

Reliability  (MTBF)  118  )  ~s.  58  hrs . 

Maintainability  (MTTk)  3  hours  4.2  hrs. 
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4,2.2  The  duty  cycle  c  plan  of  operational  use  for  the  radar 
will  be  tested  against  three  standards: 

(a)  A  four-hour  period  (ncrrtinal)  comparable  to  the  normal 
duration  of  general  quarters. 

( b }  A  90-day  period  comparable  to  a  normal  patrol  or  cruise. 

v c )  A  four-year  period,,  comparable  to  the  expected  duty  tour 
between  shipyard  overhauls. 

For  the  search  mode  the  periods  of  interest  are  the  four-hour 
and  90-dav  cycles.  The  radar  will  be  operated  continuously  in 
the  search  mode,  to  decoct  the  start  of  the  attack.  When  the 
attack  has  started  the  tracking  computer  is  activated  to  provide 
tracking  on  designated  targets.  Trie  period  of  interest  in  the 
tracking  mode  is  the  probable  maximum  duration  of  an  attack. 

We  can  make  the  following  assumptions: 

(a)  The  equipment  will  be  operated  daily  at  morning  general 
quarters  and  whenever  an  unidentified  target  is  detected. 

(b)  About  six  of  the  operations  will  result  in  attack  by 
enemy  aircraft. 

(c)  The  period  of  each  operation,  scheduled  or  unscheduled, 
will  be  four  hours. 

(d)  Failures  can  occur  only  during  operating  periods. 

The  probability  that  the  radar  is  operable  in  the  search  mode  at 
any  time  a  target  might  come  within  range  is  its  availability, 

MTBF 

A  '  MTBF  +  MTTR 

assuming  that  the  radar  set  is  designed  for  negligible  preven¬ 
tive  maintenance  downtime  during  the  90-day  cycle.  Using  the- 
parameters  previously  determined,  the  expected  availability  in 
the  search  node  is  .97  5.  The  probability  that  any  target  appear¬ 
ing  will  be  detected  in  time  is  the  product  of  probabilities  for 
performance  and  availability.  This  product  is  .925.  The  relia¬ 
bility  of  the  radar,  operating  in  the  tracking  mode  for  the  four 
hours,  is  .933. 


.  .i 

if 


The  probability  that  the  radar  will  perform  its  function  of 
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detecting  the  start  of  an  attack  and  tracking  aircraft  for  the 
four  hears  is  the  product  of  these,  or  .865.  Assuming,  as  we 
did  that  six  such  attacks  would  occur  in  a  SO -day  patrol,  the 
probability  of  successfully  operating  during  six  attacks  would 
be  .42.  The  risk  '"under  the  assumption  of  six  attacks)  of  3 
chances  in  5  of  not  surviving  a  c>0-day  patrol  does  not  apper 
acceptable . 


4.2.3  Alternate  approaches:  Improving  the  reliability  of  the 
search  and  tracking  radars  by  a  factor  of  four  would  improve  the 
MTBF  in  the  tracking  mode  to  232  hours  giving  a  four-hour  relia¬ 
bility  of  .983.  This  amount  of  improvement  is  considered  within 
present  industry  capability. 


The  availability  of  the  search  radar  in  search  mode  is  improved 
to  .994.  The  improved  effectiveness  of  the  radar  becomes  .927. 
For  a  90- day  cruise  with  six  actual  attacks,  the  probability  of 
successful  detection  and  tracking  is  improved  to  .63. 


With  a  performance  improvement  in  detection  of  aircraft  at  40- 
50  miles,  the  effectiveness  will  see  a  significant  improvement. 
A  performance  improvement  to  .98,  for  example,  would  impo  >'• 
the  single  attack  effectiveness  to  .957  and  the  cruise  effect:  v 
ness  (six  attacks)  to  ,77. 


4.3  TRADE-OFF  ANALYSIS 


A  trade-off  analysis ,  showing  these  factors,  similar  to  Figure 
2-21  should  be  prepared  to  provide  clear  visibility  to  the  CNO 
of  the  cost,  schedule  and  performance  factors  to  enable  him  to 
make  a  decision,  based  on  a  solid  foundation,  as  to  which  course 
to  pursue.  Development  of  such  trade-offs  is  covered  in  chapters 
23  and  26,  With  this  minimum  level  of  detail,  an  SOR  can  be 
definitive  not  only  ol  the  performance  characteristics  bat  also 
of  die  level  of  effort  to  apply  in  the  improvement  of  reliability 
and  maintainability. 


5 .  TECHNICAL  DEVELOPMENT  PLAN 

Once  the  foundation  is  laid  for  the  development  proa’' am  through 
a  choice  of  objectives  and  the  Specific  Operational  Requirement 
issued,  the  development  of  the  plan  for  achieving  the  objectives 
is  fairly  straight  forward.  With  the  requirements  and  industry 
capability  known,  the  plan  for  dependability  requires  a  level  of 
control  adequately  identified  by  the  gap  between  present  capabil¬ 
ities  and  requirements.  The  planning  for  accomplishment,  however. 


TRADEOFF  WORKSHEET 

REQUIRED  APPROACH  A  APPROACH  B  APPROACH  C 
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is  the  CNOs  opportunity  to  judge  the  management  effectiveness  of 
the  Bureau  on  the  project.  Lack  of  confidence  in  the  outcome, 
based  on  inadequacy  of  planning,  may  still  prevent  initiation  of 
the  project.  Inadequacy  of  the  planning  presentation  in  the  TDP 
might  well  convince  the  CNO  that  the  risks  involved  are  too  great 
fcr  the  gamble.  The  TDP  must  reassure  the  CNO  that  the  Bureau 
is  aware  of  the  problems  and  is  planning  to  overcome  them.  If 
this  conviction  is  not  clear,  some  other  allocation  of  the  funds 
may  well  be  made. 
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Chapter  3 
SYSTEM  DEFINITION 

A  system  is  a  collection  of  components  that  are  made  to  operate 
together  as  a  unit.  The  term  was  originally  used  in  the  com¬ 
munication  field  to  describe  the  various  techniques  (telephone, 
telegraph,  wireless,  amplitude  modulation,  frequency  modulation , 
pulse  modulation,  etc.)  used  to  transmit  information  from  one 
location  to  another. 

Th^  term  weapon  system  has  been  used  for  a  collection  of  smaller 
systems.  For  example,  the  weapon  system  used  for  intercepting 
bombers  includes  the  interceptor,  airborne  fire  control  system, 
armament  system  and  propulsion  system  and  in  addition  a  ground- 
to-air  communication  link  and  possibly  an  automatic  landing 
system.  The  interceptor  is  only  a  part  of  a  still  larger  system 
called  the  Air  Defense  System  which  includes  the  early  warning 
systems,  anti-aircraft  weapon  systems,  interceptors  and  associ¬ 
ated  GCI  (ground  controlled  interception)  systems  and  also  the 
communication  links  which  tie  all  these  together. 

A  ship  is  a  weapon  system  in  the  sense  that  it  is  a  collection 
of  systems  which  transports  itself  plus  a  load  along  some  sea 
path  to  a  particular  destination  with  a  specific  purpose  or 
function.  The  load,  for  military  applications,  consists  of 
weapons  and  the  destination  is  some  operating  area. 

In  the  development  of  a  highly  complex  system  such  as  a  ship, 
there  is  a  major  need  to  consider  the  interrelationships  between 
systems.  Unless  such  interfaces  are  considered,  there  is  a  great 
danger  that  efforts  to  achieve  perfection  in  one  area  may  reduce 
the  overall  effectiveness  of  the  ship,  rather  than  enhan  e  it. 

It  is  often  difficult  because  of  the  broad  technical  knowledge 
required  to  know  how  to  make  compromises  judiciously.  It  will 
be  the  purpose  of  this  chapter  to  describe  some  of  the  consider¬ 
ations  that  are  involved  in  the  development  of  integrated  systems. 

1 •  DEVELOPMENT  OF  SYSTEM  APPROACH 

DEFINITION  OF  SYSTEM  TASK 

As  we  discussed  in  chapter  2,  the  selection  of  the  task  for  the 
ship  is  complex.  Each  ship  has  a  variety  of  capabilities,  cne 
or  more  defined  as  primary,  others  as  secondary.  The  primary 
Capabilities  are  based  on  requirements  for  the  class  of  ship 
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as  defined  by  the  Ships  Characteristic  Board.  Sometimes,  as  in 
the  FBM ,  the  primary  character istic  is  related  to  the  weapon. 
Again,  as  in  the  DER  it  is  related  to  detection  and  tracking 
equipment.  Or,  as  in  the  MSO  it  may  be  related  to  counter 
measures  equipment.  The  start  of  the  analysis  may  then  be  the 
identification  of  systems  that  are  used  to  effect  the  primary 
function  or  mission  of  the  type  of  ship.  The  task  of  the  ship 
is  to  support  these  primary  systems. 

L2 _ SELECTION  OF  COMPONENTS 

H'vdog  identified  the  primary  sys terns  of  the  ship,  the  next  phase 
is  ne  uetermination  of  the  systems  needed  to  support  them.  For 
a  particular  case  the^e  supporting  systems  may  include  Navigation, 
Propulsion,  Ship  Control,  Electric  Power,  External  Communications, 
Internal  Communications,  Search,  Detection,  Life  Support,  Damage 
Control . 

1^2 _ SYSTEM  OPTIMIZATION 

Having  selected  the  p>rimary  and  supporting  systems  an  analysis 
must  be  made  of  the  performance  of  this  group  to  determine  the 
ch^-'act.  ristics  which  will  result  in  an  integrated  system. 

A  useful  aid  to  thinking  at  this  stage  of  development  is  the 
system  bit  ck  diagram.  A  block  diagram  is  a  schematic  description 
of  the  way  the  system  operates.  Each  system  or  subsystem  may  be 
thought  of  as  a  box.  (Figure  3-4).  Certain  inputs,  such  as 
signals,  power,  and  decisions  are  required  to  make  it  perform 
its  function.  As  a  result  of  tne  inputs,  the  box  produces  a 
certain  output  such  as  position,  energy,  or  other  signals.  The 
relationships  of  the  output  to  the  input  is  established  by  what¬ 
ever  mechanism  is  within  the  hex.  The  effect  of  operation  of 
such  i  m jehanism  we  call  a  transfer  function.  The  box  operates 
under  the  influences  of  its  environment  which  may  have  an  effect 
^n  the  transfer  function  to  modify  the  output.  A  simple  example 
might  be  a  steam  generator.  On  the  provision  of  fuel  and  air 
(properly  combined  and  ignited,  of  course)  and  water,  the  steam 
generator  produces  a  flow  of  steam.  The  flow  is  controlled  by 
variation  in  demand  and  the  quantity  of  fuel  and  air  burned. 

The  transfer  function  includes  the  transfer  of  heat  energy  to 
the  water. 

A  system  .  usists  of  a  number  of  such  boxes.  To  describe  the 
:-incti  a’ng  of  the  ship,  the  systems  are  arranged  in  blocks  with 
co!  e  i  i  nq  ir.es  to  illustrate  the  flow  of  information.  A 
t •  ;al  oJock  diagram  for  a  generalized  weapon  system  is  ilius- 
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trated  in  Figure  3-6. 

There  are  secondary  relationships  involved  in  these  systems. 
Navigation  may  support  the  weapon  directlv.  Communications 
provides  the  input  to  ship  control.  Electric  power  supports  each 
other  systems  and  is  itself  supported  by  elements  of  the  propulsion 
system . 

A  complete  diagram  (usually  much  detailed  than  that  illus¬ 

trated  in  Figure  3-6)  showing  all  of  the  relationships  is  some¬ 
times  called  the  interaction  diagram.  The  functioning  of  each 
system  is  described  by  the  transfer  functions  which  relate  the 
outputs  of  each  of  the  blocks  i.n  the  diagram  to  its  various  in¬ 
puts  . 

In  the  conceptual  phase  of  design,  these  systems  are  not  yet 
identified  as  specific  pieces  .of  hardware.  Their  inclusion  as 
systems  only  identifies  that  "hardware"  systems  are  needed  to 
perform  the  functions.  These  functions  need  to  be  described  to 
a  greater  level  of  detail,  breaking  the  functions  into  subfunctions 
and  these  lower  into  sub  subfunctions  until  a  conceptual  equipment 
or  assemblage  can  be  named  that  can  perform  the  function.  For 
example,  Navigation  might  include  identification  of  position  on 
the  sea  and  identification  of  true  North.  It  might  also  require 
information  on  the  speed  of  the  ship  through  the  water.  Several 
equipments  can  be  named  capable  of  performing  these  subfunctions. 

For  position  on  the  sea,  Celestial  navigation,  LORAN  or  SINS 
might  be  considered,  for  identification  of  direction  a  gyrocompass, 
for  speed  a  pitometer  log. 

The  output  of  each  such  proposed  system  operating  within  its 
intended  environment  must  be  tested  against  the  input  require¬ 
ments  for  all  related  systems.  The  trade-off  is  made;  selecting 
the  optimum  systems  capable  of  meeting  all  requirements  from 
considerations  of  cost,  schedule  and  performance  parameters.  The 
performance  parameters  should  at  this  time  include  weight,  space, 
speed,  accuracy,  reliability,  maintainability,  availability,  etc. 

One  very  good  way  to  accomplish  this  is  to  select,  first,  the -key 
properties  of  the  systems,  s  :ch  as  response  time,  voltage,  pressure, 
ere.,  which  approx  mate  ly  describe  the  operation  of  the  system. 

These  are  compared  and  values  of  the  key  properties  selected  which 
produce  the  best  performance  of  the  system  task,  accounting  for 
the  performance  parameters  named  above. 

Variations  in  the  performance  parameters  are  then  tested  against 
cost  and  schedule,  keeping  the  key  properties  within  permissible 
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limits  to  arrive  at  an  optimized  system. 

1.4  SUBSYSTEM  DEFINITION 

Having  identified  system  function  to  general  types  of  hardware 
capable  of  performing  the  functions,  the  next  step  is  to  eval¬ 
uate  the  type  of  hardware  to  define  the  characteristics  of  the 
systems  to  be  used.  Working  within  the  constraints  already 
selected,  the  input  ;:nd  output  for  each  system,  the  function 
assigned  that  system  is  again  broken  down,  one  level  at  a  time 
until  the  sub functions  derived  are  each  identifiable  to  an  equip¬ 
ment.  For  example,  "external  communication  for  the  command  net," 
may  be  broken  down  into  transmit  and  receive,  requiring  a  trans¬ 
mitter  and  a  receiver.  Transmitting  equipment  may  be  available 
that  meets  the  key  properties  and  required  performance  character¬ 
istics  already  selected.  If  not,  one  can  be  synthesized  by  break¬ 
ing  down  the  function  to  the  next  level  (Transmit  to  generate  a 
carrier,  modulate,  amplify,  and  radiate).  Working  within  the 
constraints  imposed  by  the  key  properties,  the  components  cap¬ 
able  of  performing  these  functions  are  tested  against  the  per¬ 
formance  parameters.  The  new  performance  parameters  are  then 
tested  at  the  next  higher  level  of  functional  breakdown  to 
determine  their  effect  on  the  optimization  of  the  system.  This 
successive  breakdown  to  finer  levels  continues  until  each  system 
is  defined  in  terms  of  components  accepted  as  within  the  state 
of  the  art. 

Using  the  developed  values  of  the  key  properties,  tentative 
specifications  are  made  and  designs  and  drawings  are  made  and 
new  estimates  of  the  performance  parameters  are  made.  These  new 
estimates,  plus  additional  parameters,  are  fed  back  into  the 
analysis  while  hardware  construction  is  proceeding.  Tests  on 
the  development  hardware  are  made  and  these  results  are  fed  back 
into  the  analysis.  A  flow  chart  illustrating  the  entire  operation 
appears  in  Figure  3-8.  The  information  may  flow  continuously  but 
the  configuration  can  only  proceed  in  steps  for  compatibility 
reasons. 

The  important  concept  here  is  the  system  Approach.  The  relation¬ 
ships  between  systems  is  continuously  used  as  a  control  on  the 
definition  of  the  system.  The  approach  insures  compatibility 
and  enables  system  optimization. 

2.  REPRESENTATION  OF  SYSTEMS  BY  MODELS  / 

The  system  block  diagram  provides  the  basic  skeleton  for  the 
system  model.  A  model  is  an  analytical  representation  of  the 
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system  in  terms  permitting  assessment  of  the  characteristic  of 
interest.  It  describes  what  the  system  is;  what,  how  and  vAien 
it  does  it;  and  what  external  influences  affect  it.  It  contain* 
descriptive  data  regarding  the  system  permitting  evaluation  of 
the  characteristic  of  interest  when  performance  data  is  applied. 

2.1  MODELS  AND  THEIR  PURPOSE 

The  system  model  is  the  means  by  which  relevant  information  1* 
utilized  in  an  organized  manner  to  formulate  estimates  concern-* 
ing  the  system.  The  model  makes  it  possible  to  evaluate  system 
performance  with  regard  to  a  characteristic  prior  to  actual  pro¬ 
duction  of  the  system.  Perhaps  more  importantly,  the  model 
approach  provides  the  means  of  evaluating  the  effects  of  design 
and  development  decisions  cn  the  system.  This  provides  a  sound, 
rational  basis  for  design  trade-off  studies,  design  selection 
and  parts  selection.  Finally,  by  means  of  a  model,  critical 
portions  of  a  system  in  development  or  in  use  are  identified. 

From  this  knowledge  the  needs  for  further  development  may  be 
defined. 

A  model  always  expresses  quantitative  output  (such  as  system 
reliability)  as  a  function  of  component  inputs  (such  a*  failure 
rates);  accounting  for  all  relationships  between  components 
(configuration) . 

An  important  application  of  the  model  is  in  total  system  trade¬ 
offs  to  obtain  an  optimum  balance,  within  the  mission  and  per¬ 
formance  envelopes,  between  total  cost,  schedule  and  operational 
effectiveness  of  the  system  as  discussed  in  Chapter  26. 

Models  such  as  these  permit  the  design  engineer  to  simulate 
alternative  approaches,  such  as  configurations  or  redundancy,  to 
determine  the  probable  effect  upon  Effectiveness  an d  Cost.  This 
provides  a  much  sounder  basis  for  trade-off  than  does  intuition. 

It  is  obvious  that  a  model  can  be  made  extremely  complex  and 
detailed.  In  a  detailed  form  it  contains  functional,  analytical 
and  logic  block  diagrams,  environmental  profiles,  mission  pro¬ 
files,  a  list  of  ground  rules  and  assumptions  and  a  complete  Mt 
of  equations.  But  the  model  need  not  be  extremely  complex.  A 
simpler  model,  consisting  of  a  simple  diagram  and  a  few  equations 
is  adequate  for  many  purposes.  The  model  is  a  tool  of  design 
and  should  be  no  more  complex  than  necessary  to  serve  the  iwsed- 
iate  purpose.  More,  it  should  be  kept  flexible  90  that  as 
additional  knowledge  becomes  available,  it  can  be  ;*dded  to  tho 
basic  skeleton,  with  no  reconstruction  except  as  required  to 
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incorporate  changes. 

For  major  system  trade-off  studies  prior  to  design  or  for  pre¬ 
design  apportionment  the  model  will  treat  the  system  at  the  major 
functional  block  level  as  was  done  in  Figure  3-6.  General 
assumptions  providing  great  simplification  should  be  used  at  this 
point.  Such  assumptions  are  just  as  valid  as  those  requiring 
elegant  mathematical  treatment,  because  of  the  lack  of  detailed 
knowledge  of  the  system.  All  that  is  required  at  this  point  is 
a  first  approximation  of  functional  block  values.  These  will 
suffice  for  making  the  major  trade-off  decisions  to  establish 
the  optimun  feasible  set  of  system  requirements  and  to  establish 
general  design  and  development  approaches. 

As  design  progresses  and  more  detailed  knowledge  of  the  system 
is  made  available,  the  model  will  evolve  in  detail  and  refine¬ 
ment.  For  prediction  and  especially  for  demonstration  the  model 
should  reach  the  degree  of  refinement  permitted  by  program  and 
data  constraints.  Such  a  model  will  provide  a  basis  for  selection 
of  design  and  parts. 

The  importance  of  a  model  lies  not  in  the  absolute  values  of 
numerics  that  it  generates,  but  rather  in  the  discipline  of 
analysis  and  comparative  analysis  that  it  provides.  The  model 
is  a  frame  of  reference  within  which  certain  quantities  are 
measured.  These  measurements  will  provide  fair  or  poor  approxima¬ 
tions  to  the  true  values,  depending  on  the  completeness  of  the 
model  and  the  quantity  and  quality  of  data  supplied. 

But  the  measurements  have  a  great  deal  of  relative  accuracy. 

7,’his  permits  comparisons  to  be  made  for  the  purposes  of  measuring 
progress  and  growth  and  of  making  trade-off  decisions.  Because 
a  system  model  is  necessary  to  arrive  at  major  trade-off  decisions 
and  to  establish  system  requirements,  development  and  use  of  the 
model  makes  it  possible  to  measure  progress  in  achievement  of  these 
requirements.  This  process  requires  apportioning  the  requirements 
at  the  proper  level  of  details  and  retaining  the  same  general 
ground  rules  contained  in  the  original  statement  of  requirements. 

At  the  detail  level  problems  are  detectable  by  comparing  measure¬ 
ments  with  requirements.  Effects  of  corrective  action  can  be 
evaluated.  Although  the  true  value  of  some  characteristics  or 
parameters  may  never  be  accurately  known,  the  model  allows  useful 
measurements  of  these  by  comparison. 

It  is,  therefore,  not  all-important  that  a  specified  requirement 
be  accurate  in  an  absolute  sense,  but  rather  that  it  be  stated  in 
terms  that  its  achievement  is  measurable  within  program  constraints. 
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SYSTEM  MODEL  ELEMENTS 


2.2.1  Mission  Objectives  and  Requirements:  A  system  comas 
into  being  as  a  result  of  some  operational  requirement.  A 
function  has  to  be  performed,  and  a  system  is  designed  and  pro¬ 
duced  to  perform  the  function.  The  system  model  begins  with  a 
mission  objective,  that  is,  a  statement  of  the  operational 
requirement.  This  requirement  might  be  detecting  and  tracking 
a  target,  propelling  a  ship  or  any  other  objective. 

In  order  to  meet  these  objectives  certain  functions  must  be 
performed.  To  detect  and  track  a  target  a  radar  system  might 
be  selected.  If  a  radar  system  is  to  be  used  for  this  purpose 
the  sub functions  might  include  control,  generation  of  an 
electrical  pulse,  radiation  of  the  wave,  receiving  the  return 
wave,  separating  the  incoming  wave  from  the  transmitted  wave, 
synchronizing  a  display  with  the  outgoing  pulse,  abstracting 
information  to  provide  input  to  the  tracking  computer  and  dis¬ 
plays.  Equipment  can  in  general  be  named  to  per  form  each  of 
these  functions.  (Figure  3-12). 

Therefore,  each  function  in  the  system  is  identified  with  a  class 
of  hardware  to  the  extent  required  to  estimate  the  magnitude  of 
the  development  effort.  In  a  propulsion  system  ter  example  there 
is  a  fuel  supply,  burners,  air  supply,  boilers  (including  pre¬ 
heaters,  superheaters,  etc.)  steam  lines,  turbines,  condensers, 
feed  subsystem,  reduction  gears,  screws  and  shafts.  Each  of 
these  can  be  identified  in  some  manner  with  one  of  the  functions 
required  for  propulsion.  A  state-of-art  constraint  has  now  been 
imposed,  in  that  general  design  approaches  are  now  defined.  No 
particular  boiler  has  been  specified,  but  the  system  will  have 
at  least  one  boiler,  which,  together  with  the  accessories,  will 
perform  the  function  of  energy  conversion. 

2.2.2  Event  Sequencing  and  Operating  Times:  Now  that  system 
functions  are  identified  and  schematically  related,  a  sequence 
or  set  of  sequences  of  activities  necessary  to  complete  the 
mission  objectives  must  be  defined. 

Along  with  the  sequence  of  activities,  the  necessary  system 
operating  times  are  determined.  This  enables  a  time  line 
analysis  of  the  mission  to  be  made.  Mission  activities  are 
appropriately .spaced  on  the  time  line  to  determine  the  ope.dting 
state  or  mode  of  the  sys.tem  at  any  given  time.  From  the, system 
time  analysis  the  periods  of  operation  for  each  functionfil  block 
is  determined.  The  periods  of  operation  (duty  cycle)  for  each 
functional  block  are  represented  by  time  lines  plotted  on  a  scale 
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ot  mission  time  with  activities  shown.  Some  of  the  mission 
activities  may  occur  at  random  times.  This  '-.'ill  be  quite  common 
for  shipboard  situations.  It  would  be  impossible  to  formulate 
an  exact  schedule  of  events  for  many  shipboard  systems.  This 
should  not  present  much  difficulty.  A  sequence  is  necessary  only 
to  i^rovide  duty  cycle  times  for  equipment  as  a  reference  for 
requirements  analysis.  A  typical  duty  cycle  should  be  assumed 
and  used  for  the  analysis. 

2.2.3  System  Operation:  The  time  line  analysis  already  per¬ 
formed  provides  the  basis  for  the  study  of  system  operation.  The 
activities  may  affect  operating  modes  in  a  manner  prescribed  by 
the  function  affected. 

The  radar  system  has  a  specified  mission  of  90  days  of  surveill¬ 
ance  with  a  four-hour  tracking  period  upon  target  acquisition. 
(Figure  3-14) .  Detection  of  a  target  is  a  random  event,  so  that 
frequency  of  tracking  periods  is  indeterminate.  This  makes  no 
difference  because  all  system  elements  operate  the  same  during 
surveillance  or  tracking. 

For  the  moment  let  us  assume  that  the  computer  in  our  radar 
system  is  kept  off  during  surveillance  mode.  Then,  when  a 
target  is  acquired  the  computer  is  switched  on.  It  is  kept  on 
until  completion  of  tracking,  after  which  it  is  switched  off  for 
system  return  to  surveillance  mode.  The  time  line  for  the  com¬ 
puter  would  show  this  intermittent  operation  as  a  blank  during 
surveillance  and  a  line  for  the  length  of  the  tracking  operation 
( four  hours) . 

2.2.4  Environmental  Profile:  A  descripcion  of  all  critical 
environments  as  functions  of  time  for  a  system  mission  is  called 
an  environmental  profile.  On  board  ship  it  is  not  always  feasible 
to  consider  environmental  levels  as  a  function  of  time  unless  the 
environment  is  the  predictable  result  of  a  pattern  of  equipment 
operation.  Many  changes  in  natural  or  operational  environments, 
such  as  temperature,  ship  motion,  etc.,  occur  randomly.  Frequently 
the  most  practical  way  to  consider  environmental  levels  on  board 
ship  for  estimation  of  reliability  is  to  assume  them  to  be  con¬ 
stant.  The  system  will,  of  course,  be  designed  to  withstand  the 
most  damaging  operational  levels.  But  for  reliability  analysis, 
the  assumed  environmental  level  should  be  an  average  value,  some¬ 
where  between  most  benign  and  most  severe,  according  to  the  anti¬ 
cipated  frequency  distribution  of  levels.  The  assumption  of  a 
single  value  represents  a  simplification  in  the  model  affecting 
the  accuracy  of  the  estimated  absolute  reliability.  However, 
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assumption  of  properly  chosen  constant  environmental  levels  makes 
the  model  sufficiently  useful. 


One  example  of  a  system  whose  environment  is  a  function  of  time 
is  an  underwater  television  system.  As  the  system  is  lowered  in 
the  ws<ev,  the  ambient  pressure  increases  at  a  rate  proportional 
to  the  rate  of  descent.  The  portion  of  the  environmental  profile 
describing  pressure  would  indicate  pressure  increase  to  a  maximum, 
a  constant  for  some  required  time  at  maximum  depth  and  decrease 
as  the  system  is  raised,  all  as  a  function  of  time.' .  It  is  nec¬ 
essary  to  recognize  the  fact  that  this  variation  in  pressures  will 
be  present.  But  for  the  purposes  of  reliability  estimation,  un¬ 
less  the  failure  pattern  for  the  system  can  be  determined  as  a 
function  of  varying  pressure,  a  constant  pressure  will  be  assumed. 
This  will  probably  be  chosen  as  an  average  maximum  level.  The 
environmental  profile  will  be  used  in  this  case  as  a  check  to 
assure  that  the  problem  of  varying  pressures  has  been  taken  care 
of  by  design. 

The  environmental  profile  for  the  radar  system  might  look  some¬ 
thing  like  the  following: 


Electrical  Subsystem: 

Temperature 
Relative  Humidity 

Salt  Atmosphere,  concentration,  etc. 


Range 

60°  to  95°F 
20?4  to  80% 


Nominal 

80°F 

60% 


Antenna  Subsystem: 

Temperature  -20°  to  120°F  10°F 

It  is  necessary  to  state  the  ranges  of  values  so  that  design  can 
be  checked  at  significant  points  within  the  ranges.  Nominal 
values  are  chosen  for  use  in  making  reliability  estimates,  as 
previously  discussed. 


2.2.5  Success/Failure  Criteria:  As  discussed  earlier,  a  system 
has  one  or  more  objectives  to  fulfill.  In  order  to  fulfill  its 
objectives  it  is  necessary  that  the  set  of  measurable  system  out¬ 
puts  conform  to  a  respective  set  of  tolerances.  These  tolerance 
bands  need  not  be  equal  to  the  designed  tolerance  ranges  but 
should  certainly  include  them.  The  ranges  of  required  output 
values  may  shift  a3  a  result  of  a  change  of  mode  of  system  opera- 
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tion,  as  implied  in  the  discussion  of  system  operation.  The  same 
is  true  of  input  values  as  well. 

If  a  system  whose  inputs  and  environments  are  within  ranges 
specified  for  successful  system  operation  performs  an  entire 
mission  with  its  outputs  remaining  within  tolerances  required 
for  successful  performance,  a  success  is  scored.  If  the  inputs 
and  environments  are  within  specified  ranges  and  any  system  out= 
put  deviates  from  its  acceptable  range  at  any  time  during  the 
mission.  At  is  considered  a  failure.  If  the  system  is  restored 
to  operation  within  a  specified  allowable  repair  time  a  mission 
success  may  still  be  achieved.  Otherwise  it  will  be  scored  a 
failure.  If  any  input  deviates  from  its  specified  range  the 
result  should  be  scored  "no  trial",  unless  it  is  conclusively 
shown  that  the  apparent  success  or  failure  would  have  resulted 
without  the  input  deviation.  It  is  evident  that  in  treating 
reliability  data  it  is  just  as  important  to  consider  input  con¬ 
ditions  and  environments.  These  system  inputs  affect  system 
performance  as  much  as  do  the  outputs.  An  obvious  case  is  a 
deck  winch  attempting  to  hoist  too  large  a  load.  On  the  other 
hand,  a  PPI  scope  may  be  presenting  an  acceptable  display,  as 
measured  in  millilumens,  only  because  the  ship’s  a-c  voltage 
is  too  high.  Such  occurrences  often  result  in  false  reporting 
or  non-reporting  of  failures. 

The  probability  that  the  system  will  be  able  to  meet  the  success 
criteria  depends  on  how  stringent  the  criteria  are.  The  criteria 
are  established  from  the  system  objectives  and  are  the  limits  of 
acceptable  ranges  of  operation  as  previous  described.  In  order 
to  obtain  uniform  reliability  estimates,  these  criteria  must  be 
stated.  They  are  also  required  in  order  to  collect  reliability 
attribute  (success/Failure)  data  or  interpret  variables  (output 
values)  data. 

For  the  radar  system  under  study,  the  success  criteria  might 
proceed  as  shown  below  in  incomplete  form: 

Output  Pulse: 

Power  8  magawatts  min. 

Frequency  2198  +.  5  me 

Pulse  Width  -  etc. 

Input  sensitivity  at  2198  +  5  wc?  20  mv  max. 
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Tracking  Accuracy : 

Range  £0.5 

Bearing  £5° 

Course  -  etc. 

These  criteria  are  not  necessarily  equal  to  performance  specifi¬ 
cations  but  the  specified  tolerance  ranges  must  be  included  with¬ 
in  the  success  criteria.  If  the  product  meets  all  of  the  complete 
set  of  success  criteria,  this  is  no  absolute  guarantee  that  the 
system  can  detect  and  track  targets  100%  of  the  time  chat  the 
system  is  so  operating.  A  small  target  in  rough  seas  may  not  be 
successfully  tracked,  or  a  target  might  not  be  detected  in  heavy 
fog.  This  is  no  reflection  on  the  reliability  of  the  system. 

It  simply  is  not  designed  to  cope  with  these  situations.  In 
other  words,  the  input  conditions  in  these  cases  of  apparent 
system  failure  are  not  as  specified.  Therefore,  though  the 
system  fails  to  meet  its  objective,  it  doe3  not  fail  in  its  per¬ 
formance.  Success  or  failure  depends  both  on  input  and  output 
per formance. 


3  * 


LOGIC  BLOCK  REPRESENTATION 


Up  until  this  point  the  model  elements  discussed  are  those  re¬ 
quired  for  making  any  kind  of  rational  system  analysis.  These 
elements  are  mission  objectives,  functional  flow  diagrams,  time 
analyses,  description  of  system  operation,  environmental  profile 
and  success/ failure  criteria.  Some  degree  of  information  is 
required  for  each  of  these  elements.  The  accuracy  of  the  analysis 
depends  on  the  accuracy  and  completeness  of  the  input  data  for. 
each  element.  How  these  data  are  integrated  in  the  model  will 
be  shown. 

3.1  SIMPLIFICATIONS 


In  the  discussion  of  the  generalized  model,  it  was  shown  that 
the  exercise  of  a  mode"!  requires  detailed  data  regarding  the 
equipments,  environment  and  interrelationships.  All  quantities 
in  the  model  are  time  dependent  distributions.  Due  to  model 
complexity  and  lack  of  accurate  data,  it  was  further  shown  that 
such  a  model  is  difficult  to  handle  in  most  cases.  Simplifying 
assumptions  which  degrade  model  accuracy  were  given.  Some  oi: 
these  are  restated  here: 

(a)  Drift  failures  may  be  neglected; 


i 
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(fc)  System  elements  are  considered  to 


sent  r  v ; 


(c)  A  failure  of  any  system  element  is  considered  to  result 
in  inevitable  system  failure,  unless  ar,  alternate  element  or 
procedure  can  supply  the  failed  function. 


These  assumptions  maxe  it  possible  to  construct  a  logic  diagram 
of  a  system,  from  the  functional  block  diagram.  The  logic  dia¬ 
gram  describes  the  system  operation  in  simple  terms,  allowing 
immediate  derivation  of  a  system  reliability  equation.  Several 
forms  of  logic  diagrams  are  available,  including  one  that 
utilizes  symbols  of  gate  functions  and  time  delays.  One,  that 
is  fairly  easy  to  set  up  and  understand,  shows  all  possible 
alternate  ways  that  the  system  can  perform  its  function.  This 
j.s  illustrated  in  Fiaure  3-19. 


3.2  APPLICATION  TO  RELIABILITY 

Estimation  of  reliability  requires  the  use  of  a  model  to  describe 
the  system  in  terms  such  that  application  of  available  reliability 
data  produces  the  required  reliability  evaluation.  The  model 
should  _e  only  as  complex  as  the  system  and  its  requirements 
demand.  It  will  range  from  a  single  function  block  with  a  single 
equation  to  a  set  of  complex  detailed  logic  diagrams  with  an 
elaborate  computer  program.  Very  complex  models  are  seldom  re¬ 
quired  for  shipboard  systems. 

To  provide  useful  reliability  measurement,  the  model  must  be 
applied  consistently.  An  assumption  used  for  apportionment  must 
also  apply  to  prediction,  demonstration  analys;s  and  support 
analysis.  During  design  and  development  it  is  frequently 
necessary  to  alter  initial  assumptions  as  more  knowledge  of  the 
system  is  gained.  The  consistency  requirement  of  the  model  means 
that  the  latest  changed  body  of  assumptions  be  applied  to  analyses 
previously  performed,  if  the  results  of  these  analyses  are  to  be 
applied  in  the  future.  The  purpose  of  consistency  is  to  permit 
valid  comparisons  to  be  made.  The  measured  achievement  must  be 
in  the  same  terms  as  the  requirement.  Measured  growth  must  be 
consistent  with  the  period  to  which  the  growth  is  referred.  The 
value  of  the  quantitative  approach  to  reliability  is  realized  only 
when  those  measurements  are  made  within  the  same  frame  of  reference. 
It  is  this  consistency  that  provides  discipline  in  reliability. 

The  reliability  analyses  model  can  be  expanded  to  include  other 
characteristics  or  program  constraints  for  trade-off  analysis  as 
discussed  in  Chapter  25  and  26.  The  trade-off  analyses  provide 
an  insight  ito  the  contractor's  understanding  of  the  development 


RADAR  LOGIC  DIAGRAMS 


and  its  problems .  In  the  tin  bi  :•.  r  sr.  ulu  c-* 

a  conceptual  reliability  node  1  surficent  t  iniini  >  -.t*r  pr 

to  anticipated  develcorr.ent  prc-b  Iem* .  The  rs.nl  trade-'.' if 
voive  reliability  and  sr.»  int  a  inab  i  1  i  ty  a  ithin  the  constraint:; 
imposed  by  development  budget  and  schedule,  product  ion  costs  and. 
field  support  of  the  system.  thorough  understand!  no  •  f  toe 
constraints  and  their  effects  is  room  red  for  valid  trade  off. 

The  amount,  of  effort  for  trade-off  is  determined  by  the  love  1  of 
reliability  required  and  the  severity  of  the  constraints  relative 
to  complexity  of  development . 

4  •  APPLIC  AT  ION  OF  THE  MODEL  TO  _  DEC  IS  ION  MAKfhd 

As  the  design  gets  underway,  the  Criteria  and  Conf  !  duration  per¬ 
mit  selection  of  an  appropriate  model  technique  from  the  avail¬ 
able  tools.  The  model  is  constructed  by  the  des  ion  one  incur,  or 
by  reliability  engineers  if  so  delegated.  As  tentative  design 
environmental  and  sequence  information  is  developed,  and  best 
available  data  (such  as  failure  rates)  is  plugged  into  the  model, 
it  can  then  begin  to  provide  useful  output. 

Although  an  apportionment  of  reliability  af’  other  design  assurance 
elements  may  have  been  made,  the  first  mode,  prediction  is  a  far 
sounder  apportionment.  Any  discrepancy  between  predicted  and 
required  value"  can  be  apportioned  rationally  on  the  model  struct¬ 
ure,  If  rational  apportionment  to  lower  levels  will  not  reach 
achievable  values,  the  design  engineer  has  a  problem. 

From  this  time  on,  the  model  is  used  for  regular  (such  as  bi¬ 
weekly)  predictions  of  reliability  and  other  design  assurance 
parameters.  Updated  plots  of  the  prediction  vs .  schedule  pro¬ 
vide  Engineering  supervision  with  regular  progress  reports.  At 
the  overall  system  levels,,  these  provide  progress  reports  to 
Management . 

Since  the  model  always  expresses  outputs  as  a  function  of 
constituent  inputs,  derivatives  thereof  may  also  be  published, 

A  Sensitivity  List  may  show  the  ratio  of  output  improvement  to 
an  arbitrary  improvement  of  each  component  input  (such  as  halving 
the  failure  rate),  in  rank  order  of  potential  improvement.  Thus 
the  design  engineer  can  quickly  spot  the  best  opportunities  for 
improvement . 

Another  derivative  is  the  Criticality  List,  which  ranks  components 
in  the  order  of  probability  of  causirg  system  failure,  taking 
failure  modes  and  effects  into  account.  This  provides  a  basis 
for  design  review,  critical  component  identification  and  action, 
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and  special  handling. 

As  design  problems  are  brought  imo  focus,  by  the  model  or  other¬ 
wise,  the  model  can  be  used  to  evaluate  alternative  solutions. 

The  design  engineer  fed)  substitutes  alternative  configurations 
and/or  components  into  the  model,  and  lets  it  calculate  the  con¬ 
sequences.  This  is  especially  powerful  if  the  model  accounts  for 
total  cost,  as  it  provides  the  economic  basis  for  a  change. 

The  system  model  approach  is  a  methodology  designed  to  give  co¬ 
hesiveness  and  visibility  of  the  problem.  It  is  one  or  many 
methods  of  organizing  data  to  identify  the  complexities  of  inter¬ 
relationships  between  the  equipments.  In  the  Radar  example: 

(a)  The  task  was  identified  by  the  characteristics  of  the 
weapon,  since  the  weapon  identified  the  nature  of  the  turret 
and  so  disclosed  its  characteristics  and  icntified  the 
nature  of  the  operational  employment  of  the  ship.  An  alter¬ 
ation  to  the  weapon  or  a  change  in  weapons  (such  as  from 

rifles  to  missiles)  alters  the  requirement  on  the  Radar.  The 
documentation  provided  in  the  model  will  clearly  indicate  any 
need  for  change  in  the  design  of  the  Radar. 

(b)  The  operating  requirements,  with  time,  environment,  and 
failure  definitions,  described  the  equipment  adequately  to 
identify  technological  areas  where  problems  may  exist. 

(c)  The  model  identified  the  reliability  requirement,  or  a* 
least  laid  the  groundwork  for  suen  identification.  It  will 
assist  in  reliability  and  maintainability  prediction  by  de¬ 
fining  the  equipment  and  anticipated  stress  levels. 

(d)  It  will  be  used  to  design  test  and  demonstration  programs. 

(e)  During  the  entire  development,  it  provides  management 
visibility  of  the  objective  and  the  progress  toward  achieving 
that  objective. 

5 .  PERSONNEL  AS  A  SYSTEM 

The  impact  of  personnel  at  every  stage  in  the  development  of  a 
system  cannot  be  ignored.  People  design,  build,  operate  and 
maintain  the  equipment.  In  many  systems  the  subjective  eval¬ 
uation  of  the  information  provided  by  the  display  initiates  the 
succeeding  operation.  In  other  systems,  the  adequacy  with  which 
maintenance  is  performed  has  a  marked  influence  on  the  success 
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or  failure  of  the  vqu  ipr.ient . 

The  modern  "black  box"  concept  in  equipment  design  has  brought 
the  designer  face  to  face  with  the  human  operator  problem. 
he  has  to  consider  the  characteristics  of  electronic  'black  boxes" 
sc  must  he  consider  the  human  "black  box" .  Each  unit  in  a  system 
will  accept,  only  certain  inputs  and  emit  only  certain  outputs  and 
each  in  turn  will  operate  satisfactorily  only  when  used  within 
given  tolerances.  The  human  component  is  no  exception.  In  order 
to  obtain  reliable  human  performance  in  a  man-machine  system, 
the  man  must  be  able  to  work  within  his  characteristic  tolerances , 
The  design  of  the  equipment  he  operates  must  match,  his  impedances 
at  both  input  and  output  stage. 

As  a  "black  box'*  ,  man  can  be  represented  in  the  system  as  a  system 
element  as  shown  in  Figure  3-23  to  make  the  system  complete.  Vie 
must,  in  fact,  consider  the  role  of  man  in  the  circuit.  The  of fee* 
of  man  on  the  design  process  and  his  relationship  to  the  relia¬ 
bility  of  the  system  will  be  discussed  later-  (Chapter  14) * 

6 .  REFERENCES 

(1)  System  Reliability  Engineering,  Gerald  H.  Sandler,  Prentice 
Hall  Technology  Series,  Englewood  Cliffs,  New  Jersey,  1963. 

(2)  System  Reliability  Measurement  and  Analysis,  R.  R.  Landers. 
Proceedings  of  the  Fourth  National  Symposium  on  Reliability 
and  Quality  Control,  January  1956. 

( 3 )  Some  Reliability  Aspects  of  System  Design,  F.  Moskowitz  and 
J.  B.  McLean,  IRE  Transactions  on  Reliability  and  Quality 
Control,  September  1956. 

(4)  Decision  'Theory  and  systems  Analysis,  Dr.  A.  Enthoven, 

Lecture  at  Lisner  Auditorium,  December  5,  1963. 

(5)  Prediction  of  Reliability,  J.  Connor,  presented  at  Sixth 
National  Symposium  on  Reliability  and  Quality  Control, 

January  1960. 

( 6 )  Integrating  Reliability  Progress  into  Design  and  Engineering; 
A  Study  of  Systems  Management.  F  Vel«on  and  A.  Steinberg, 
Aerospace  Reliability  and  Maintainability  Conference,  AIAA , 
SAE  &  ASME,  May  1963,  Washington,  D.  C. 

( 7 )  Integration  of  Design,  Overall  Sys tem  Cons ider at i ons , 

J.  C.  Fletcher. 


3-24 


(8 )  On  Mathematic;?  1  Modeling  <nd  Research  in  Systems,  W.  Karush , 
System  Development  Corp-  (Ar°98305] 

(9)  Modeling  and  Simulation  as  a  Research  Tool,  W .  joslyn,  Boeing 
Corp.,  ;  resented  at  the  C  OR  51 --OR  S  A  1964  Joint  Conference,  May 
1964. 

(10)  Military  Systems  Analysis,  E.  S.  yuade,  Rand  Corn.  (AD292026) 

(11)  An  Overall  Viewpoint  of  Systems  Analysis,  M.  Flood,  University 
of  Michigan,  Ann  Arbor,  Michigan,  presented  at  SAE  Annual 
Meeting,  January  1960,  Detroit,  Michigan. 

(12)  Human  Factors  in  System  Analysis,  H.  Goldhamer,  Rand  Corp., 
ASTIA  Document,  ATI  78026. 


/ 


4-1 


Chapter  4 
PROBABILITY 


a 

1  . 

SIMPLE  EVENTS 

4-  3 

1  .  1 

Denotation  of  Probability 

4-  3 

1.2 

Favorable  Outcome 

4-  4 

1.  7 

Numerical  Basis 

4-  5 

1 

Definitions  of  Related  Terms 

4-  5 

2  . 

COMPOUND  EVENTS 

4-  6 

2  .  1 

Simple  Combinations 

4  -  6 

2.2 

Complex  Combinations 

4-  8 

2  .  3 

Summary 

4-10 

3 . 

BINOMIAL  PROBABILITY  DISTRIBUTION 

4-10 

3  .  i 

Large  Numbers  of  Trials 

4-10 

3.2 

Dofiniti ons 

4-12 

5  .  3 

Binomial  Theorem 

4-12 

3.4 

Binomial  as  a  Probability  Distribution 

4-12 

4. 

EMPIRICAL  PROBABILITY 

4-13 

4  .  1 

Fundament  a  1  Concept 

4-13 

4 . 2 

Do  f i ni t ion 

4-13 

4.3 

The  pole  of  "Empirical"  Probability 

4-15 

4.4 

Assumption  of  Equally  Likely  Events 

4-16 

4.4 

Validity  of  Probability  The  tv  in  Reliability 

Eng ineer i m 

4-  16 

5.  FAT  LURK  DENSITY  FUNCT  IONS  4-17 
r>.  1  Equipment  Failures  4-17 

S.2  Use  of  Probabilistic  Mathematics  in  Reliability  4-17 


(>.  APPL  1  CAT  ION  TO  RELIABILITY  COMPUTATION  4-17 

6.1  Complete  Systems  4-19 

6.2  Pt  edict  ir.  •  Reliability  of  Systems  From  Reliability 

of  Parts  4-26 


/.  SAMPLE  COMPUTATIONS  4-27 

7.1  Series  Case  -  Normal  Distribution  4-27 

7.2  General  Series  Case  -  Normal  Distribution  4-29 

7.3  Series  Case  -  Exponential  Distribution  4-jl 


7.4  Series  Parallel  Case  -  Exponential  Distribution  4-31 

7.5  Standby  Parallel  Case  -  Exponential  Distribution  4-34 


P  . 


4-39 


“ 


REFERENCES 


4-2 


Chapter  4 
PROBABILITY 

Our  purpose  in  this  section  on  probability  is  to  acquaint  you 
with  the  basic  ideas  of  probability  underlying  c'ne  analysis  of 
reliability.  Our  goal  is  not  to  convert  engineers  into 
statisticians,  but  rather  to  create  a  common  ground  for  the 
efficient  exchange  of  ideas  concerning  reliability. 

The  word  probability  is  used  loosely  in  our  daily  conversation 
and  we  know  vaguely  what  it  means.  We  talk  of  the  probability 
of  winning  a  game  of  cards  or  dice  or  a.  football  game,  the 
probability  of  its  raining  tomorrow,  or  the  chance  of  a  person 
living  to  be  so  many  years  old.  In  all  these  cases  we  are 
interested  i..  a  future  event,  of  which  the  outcome  is  uncertain, 
and  about  which  we  want  to  make  a  kind  of  prediction.  We  would 
like  to  be  able  to  devise  a  way  of  measuring  the  probability  of 
on  event  --  not  only  to  determine  its  probability,  but  also  to 
compare  the  probability  of  different  events. 

Historically,  probability  theory  has  had  a  strong  relationship 
with  games  of  chance,  i.e.,  gambling,  such  as  roulette,  dice, 
poker,  bridge,  and  black  jack  (twenty  one).  The  one  common 
characteristic  in  all  these  games  ^ f  chance  is  the  unpredicta¬ 
bility  of  what  happens  on  a  given  deal  or  a  given  turn  of  the 
wheel,  i.e.,  on  a  given  trial.  However,  as  is  known  by  any 
individual  who  has  played  any  one  of  these  games,  there  is 
regularity,  and  hence  predictability,  in  the  course  of  a  Large 
number  of  trials.  Probability  theory  is,  in  genera),  concerned 
with  the  predictability  of  occurrences  in  a  large  number  of 
trials,  i.e.,  predictability  "in  the  long  run"  or  "on  the 
average . " 

There  are  a  large  number  of  areas  in  which  the  characteristics  - 
unpredictability  during  a  given  trial  and  predictability  over  a 
large  number  of  trials  -  can  be  found.  Probability  theory  has 
found  an  application  in  each  one  of  these  areas.  The  diversity' 
of  application  of  probability  theory  can  be  illustrated  by  list¬ 
ing  some  of  the  areas  in  which  this  theory  is  u3ed. 

1.  Theoretical  Fhysics:  Statistical  thermodynamics 
and  quantum  mechanics. 


Nuclear  Reactor  Technology:  Atomic  Bomb  development 
and  critical  sizes  of  nuclear  engines. 
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3.  Cnmmun icat ion  Theory:  Telephone  trunk  lines  and 
RF  comnunicat ion  links. 

4.  Insurance:  Life  insurance  and  automobile  accident 
insurance . 

5.  Medical  Research:  Genetics  and  the,  ~y  of  epidemics. 

6.  Theory  of  Learning 


1.  SIMPLE  EVENTS 

1.1  DENOTATION  OF  PROBABILITY 

Since  the  term  probability  is  applied  to  so  many  different 
events,  it  seems  that  one  can  hardly  give  it  a  definite  meaning 
without  some  simplification.  Later  on  we  will  see  that  we  must 
extend  the  definition  to  include  more-  complex  situations  in 
order  to  have  a  useful  theory.  So,  taking  fundamentals  first, 
let  us  see  how  much  we  may  already  know  about;  a  method  of  assign¬ 
ing  numbers  to  the  likelihood  of  chosen  events.  Consider  the 
simple  experiment.,  the  tossing  of  a  coin.  On  a  single  toss  cf 
the  coin,  there  are  only  two  outcomes  -  heads  or  tails.  Every¬ 
one  will  aqree  that  if  the  coin  is  "honest,"  i  . ,  uniformly  and 
symmetrically  made,  and  if  the  tossing  is  "fairly  done,"  there 
is  no  reason  to  expect  the  appearance  of  a  head  ary  more  than 
the  appearance  of  a  tail.  In  everyday  language,  wo  say  that 
"the  coin  has  1  chance  in  2  of  falling  heads";  in  technical 
language,  we  say  that  "the  probability  of  heads  is  1,2."  In 
symbols ,  we  write: 

P  (H)  *  1-2 

Similarly,  in  the  tossing  of  a  die,  the  face  with  six  dots  has 
1  chance  in  6  of  landing  on  top;  for  it  is  assumed  that  the  die 
is  well  made,  thrown  "fairly."  and  there  is  no  reason  for  expect¬ 
ing  any  one  face  to  turn  rather  than  any  other.  We  say  that 
"the  probability  of  6  dots  on  top  is  1/6."  In  symbols: 


Likewise,  when  we  take  a  card  from  a  well-shuffled  bridge  deck, 
we  have: 

P  (A)  =  ,/5.2 
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1 . 2  FAVORABLE  OUTCCME 

Now ,  as  an  experiment,  consider  a  10 -ticket  draw  for  a  p*-ize.  A 
name  is  written  on  each  of  10  tickets,  ♦■he  tickets  are  then 
thoroughly  mixed  in  a  bag,  and  1  ticket  is  drawn.  The  person 
whose  name  appears  on  the  ticket  so  drawn  is  the  winner. 

If  Susie's  name  appears  on  -just  1  ticket,  her  chance  of  winning 
the  prize  is  1  in  10,  since  all  outcomes  in  the  drawing  are 
"equally  likely, "  that  is,  one  of  the  10  tickets  shall  be  drawn, 
and  there  is  no  reason  for  expecting  any  1  ticket  to  be  drawn 
rather  than  any  other.  Thus, 

P  (s)  -  1/10 

Similarly,  if  Susi ?’ s  name  appears  on  7  tickets,  her  chance  of 
winning  is  7  in  10,  and 

p  (S)  -  VIO 

The  general  idea  is  that  of  separating  from  the  whole  set  o. 
equally  likely  outcomes,  the  special  subset  of  favorable  out¬ 
comes  .  We  use  the  term  "set'  to  speak  -t  a  group  of  anything 
with  a  particular  characteristic ,  as  the  "set"  •> f  all  American 
males.  Wo  use  the  term  "subset"  to  mean  a  group  completely  w  th 
in  the  set.  with  some  other  character ist i c  as  the  subset  of 
"red-haired  American  males.' 


The  probability  of  a  favorable  outcome 
following  rule: 

r,  ,  ,  „  ,  ,  ,  number  of 

P  ( favorable  outcome)  -  - 

numoor  of 


is  ass igno a  b v  t h <■' 

favorable  ou to ame u 
COS  6  1  D  1  4 v  • U  t  OOfo t?  s 


This  method  of  assign  mo  to  a  favorable  •outcome  a  measure,  or 
number,  called  its  prob  »bi  1  lty ,  has  an  immediate  consequence , 
for  if  there  a^e  no  favorable  c  ; tcoir.es  in  tnc  set  ot  poss  i  b  .e 
outcomes,  then 


P  (  t.  a  v or  a b  .1  e  o u  t  c < *ae}  -  0 
and ,  if  all  possible  (.outcomes  are  favorable,  then 

P  ( 1  a v or.? b  1  e  o u t e am e )  1 


It  follow's  that 


0 


F  ( favorable  . ut come) 


•>  1 
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that  is,  the  probability  of;  a  favorable  outcome  lies  within  the 
range  of  numbers  from  zero  to  1.  For  if  a  favorable  outcome  is 
certain  to  happen,  its  probability  is  1,-  if  it  is  certain  to 
fail,  its  probability  is  0;  in  every  other  case  its  probability 
must  be  between  0  and  1. 

Another  consequence  of  this  method,  since  every  outcome  must  be 
either  favorable  or  unfavorable,  is 

P  (favorable  outcome  )  +  P  (unfavorable  outcome)  =  1 

1.3  NUMERICAL  BASIS 

The  definition  of  probability  is  based  on  a  count  of  the  number 
of  possible  results  of  a  trial.  Since  there  are  six  possible 
outcomes  of  a  throw  of  a  single  die,  the  probability  of  any  one 
number  occurring  is  one  sixth,  assuming  no  bias.  In  general,  if 
an  event  can  occur  in  m  ways-  and  can  fail  in  n  ways,  the  proba¬ 
bility  of  its  occurrence  is  m/(m  +  n) ,  provided  the  ways  are 
exhaustive,  equally  likely,  and  mutually  exclusive. 

The  concept  "equally  likely"  is  basic?  it  can  play  the  role  of 
the  undefined  element.  Indeed,  it -is  quite  difficult  to  define 
the  term  "equally  likely"  without  using  the  word  probability. 

The  term  can  be  described  as  being  the  lack  of  any  bias  favoring 
one  way  over  another  in  the  random  trial. 

The  ways  are  "mutually  exclusive"  if,  when  one  is  known  to  occur, 
the  other  is  known  not  to  occur.  For  instance,  if  the  event  is 
the  drawing  of  a  single  card  from  a  deck  and  obtaining  an  ace, 
there  are  four  mutually  exclusive  ways  of  doing  it:  by  drawing 
an  ace  of  spades,  hearts,  diamonds,  or  clubs.  If  a  single  card 
is  drawn  and  it  is  the  ace  of  spades,  it  cannot  be  the  ace  of 
another  suit.  Here  the  ways  are  equally  likely  if  the  drawing 
procedure  is  not  biased  in  favor  of  any. one  card.  The  proba¬ 
bility  of  drawing  an  ace  is  4/52  under  these  conditions. 

1.4  DEFINITIONS  OF  RELATED  TERMS 

Some  of  the  terms  which  were  used  in  the  definitions  of 
probability  and  some  other  terms  which  will  be  needed  later  are 
defined  as  follows: 

1.  Exhaustive:  As  used  in  the  definitions  of  probability, 
the  term  "exhaustive"  means  that  all  possible  ways  for 
an  event  to  happen  are  included.  The  reasons  for  this 
restriction  in  the  definition  should  be  obvious. 
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2.  Trial  i  Each  attempt  under  a  certain  set  of  rules  to 
produce  an  event  A  (where  the  outcome  of  the  event  A  is 
uncertain)  is  a  trial.  Thus,  each  repeated  throw  of 
dice  in  a  game  of  dice  is  an  attempt  to  make  one's 
point.  One  usually  speaks  of  a  random  trial.  The  term 
"random"  implies  "without  bias." 

3.  independent  Trials;  If  the  outcome  of  one  trial  does  not 
influence  the  outcome  of  a  subsequent  trial,  the  two 
trials  are  said  to  be  independent.  Each  throw  of  dice  in 
a  dice  game  meets  this  criterion.  However,  the  drawing 
of  cards  from  a  deck  without  replacement  does  not  meet 
it,  since  the  number  of  ways  an  event  (drawing  a  speci¬ 
fied  card,  for  example)  can  happen  changes  with  each 
drawing . 

4.  Conditional  and  Unconditional  Probabiliti.es:  The  con¬ 
ditional  probability  of  an  event  is  encountered  when 
information  about  the  occurrence  of  some  other  event  is 
available.  If  one  is  informed  that  a  certain  event  has 
occurred,  does  this  tell  him  anything  about  the  probabil¬ 
ity  of  the  occurrence  ■  f  another  event?  Knowing  that 

B  has  occurred,  what  is  the  probability  of  A  occurring? 

If  the  events  are  dependent,  the  knowledge  that  one  has 
occurred  does  modify  the  probability  of  the  other,  anvd 
this  probability  is  conditional .  If  no  in  format icn  is 
available  as  to  the  result  of  an  event  on  a  previous 
trial,  the  probability  is  unconditional. 

COMPOUND  EVENTS 


Sometimes  the  problem  requires  that  a  set  of  outcomes  be  con¬ 
sidered  a  sinule  event.  The  problem  of  throwing  a  six  each 
time,  on  two  successive  trials,  and  the  problem  of  drawing  four 
aces  on  successive  trials  without  replacement  are  compound 
events . 

mhe  determinat ion  of  the  probability  of  success  in  these  two 
cases  requires  some  refinement  to  our  method  of  determining 
pus  s ib 1 e  out  comes . 

2.1  SIMtLE  COMBINATIONS  : 

Expressions  of  the  form  "A  or  B"  use  the  word  "or"  ii.  two 
different  wa/s:  (1)  in  the  exclusive  sense,  which  connotes  "A 
or  B,  but  not  both"  (e.g.,  a  coin  falls  "heads  or  tails");  (2) 
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in  the  inclusive  sense,  which  connotes  "A  or  B  or  both"  (e.g., 

"The  weather  looks  as  if  it  may  sleet  or  snow."}  Ordinarily, 
the  context  is  a  sufficient  guide  to  the  intended  meaning.  When¬ 
ever  the  expression  "A  or  B"  is  used  in  referring  to  events ,  we 
will  always  use  the  inclusive  "or";  in  other  words,  event  "A  or 
B"  means  "A  or  B  or  both".  This  inclusive  "or"  is  defined  in 
probabilistic  language  as  a  "union"  of  sets. 

The  idea  of  simultaneous  manbership  in  two  sets  is  connoted  in 
our  use  of  "and"  when  we  talk  about  events.  Thus,  if  events  A 
and  B  are  subsets  of  a  set,  then  "A  and  B"  is  their  "intersection", 
that  is,  the  event  "A  and  B"  contains  those  sample  elements  that 
belong  to  both  A  and  B. 

By  using  our  definition  of  probability  (2.2),  three  rules  are 
needed  for  the  calculation  of  the  probabilities  of  compound 
events.  Let  E  and  F  denote  events;  then,  we  want  to  knew  the 
probabilities  of  the  following  events  derived  from  them: 

a.  The  event  "not  E"  (E  does  not  occur).  (or  "not  F" ) 
b»  The  event  "E  or  F"  (either  E  or  F  or  both  occur) . 
c.  The  event  "E  and  F"  (both  E  and  F  occur) . 

For  example,  the  event  "E"  may  be  "we  get  a  6",  the  event  *F" 
may  be  "we  get  a  5".  "Not  E"  is  simply  "not  getting  a  6" 

(getting  any  of  the  faces  from  1  to  5) .  “ E  or  F”  is  "getting  a 

5  or  a  &" .  "E  and  F"  is  impossible  for  a  single  throw. 

Rule  1.  The  probabilities  of  the  events  "E"  and  "not  E”  satisfy 
the  equation. 


P  (not  E)  =  1  -  P  (E) 

Example  -  Three  coins  are  tossed.  What  is  the  probability  of 
getting  at  least  one  head. 

Solution  ~  Since  the  question  as  stated  really  asks  what  is  the 
probability  of  getting  one,  two,  or  three  heads,  it  can  be  solved 
more  simply  by  computing  the  probability  of  getting  3  tails  (0 
heads).  There  are  eight  equally  likely  outcomes  (HHH,  HHT,  HTH, 
TIHI,  HTT ,  THT,  HTT  and  YTT)  .  Only  one  of  these  (TTT)  corresponds 
to  the  requirement  of  three  tails.  The  probability  of  this  is 
1/8.  This  is  "not  E" .  The  probability  of  E  (at  least  one  head) 

=  1  -  P  (Not  E)  or  7/8,  hence  P(E)  *  7/8. 

Rule  2.  If  two  events  E  and  F  are  mutually  exclusive,  then 
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P  (E  or  F)  -  P(E)  +  P(F) 

Example  -  What  is  the  probability7  that  a  card  drawn  at  random 
from  a  deck  of  cards  is  either  a  neart  or  the  queen  of  spades » 
Call  E  the  event  "heart"  and  F  the  event  "queen  of  spades".  By 
our  definition 

P (E)  ^  ~  and  P(F)  =  ~ 

52  o2 


By  rule  2 


P(E  or  F) 


13  __1  =  14 

52  +  52  52  * 


Rule  3.  If  E  and  F  are  independent  events.,  then 


P(E  and  F)  =  P(E)  x  P  ( F'5 


Example  -  From  a  deck  of  cards,  two  cards  are  drawn  at  random, 
successively;  the  first  being  replaced  before  the  second  is  drawn 
What  is  the  probability  that  th--j  first  is  a  heart  and  the  second 
is  not  a  king.  If  E  denotes  "a  heart"  and  F  denotes  "not  a 
king" . 


By  rule  3 


P(E) 


13 

52, 


P(F) 


48 

52 


P ( E  and  F ) 


13  4J3  _  _12 

52  X  52  52 


2.2  COMPLEX  COMBINATIONS 

2.2.1  Conditional  Probability:  To  introduce  the  notion  of 
dependent  events  consider  the  following  example; 

An  urn  contains  3  red  balls  and  2  black  balls.  Two  balls  are 
drawn  in  succession  without  replacement.  If  the  first  ball 
drawn  was  black,  what  is  the  probability  that  the  second  ball 
drawn  will  be  red? 

Solution:  Since  we  know  the  first  one  was  black,  v/e  are 
actually  in  a  new  situation,  -  we  have  a"  urr.  containing 
3  red  balls  and  1  black  ball;  hence  by  definition,  r (R)  -  3/4 

If  we  locus  our  attention  on  the  evert  "getting  a  red  ball",  it 
is  ckur  from  the  example  that  the  probability  of  this  event  de¬ 
pends  upon  the  information  one  has  at  hand.  The  probability 


usually  isn't  written  as  P{R)  but  rather  P  (getting  red  given 
that  black  has  occurred),  or  symbolically  r(R|B).  This  probabil¬ 
ity  P(R|B)  is  called  the  "conditional  probability  of  getting  a 
red  ball  on  the  second  draw,  given  that  a  black  one  was  drawn  on 
the  first  draw." 

We  saw  that  by  taking  into  account  what  actually  was  happening, 
P(R|B)  was  easily  calculated.  Another  way  of  calculating  P(RjB) 
is  the  following: 

Number  the  balls  rp,  r2,  r^,  b]_,  b2;  then  list  all  the  equally 
likely  outcomes  of  drawing  two  balls  from  the  urn  when  the  first 
ball  is  not  replaced.  The  20  equally  likely  cases  are: 


‘V  b2 

T  2  ‘ 

1, 

~2 

r  3  * 

fc2 

ri 

b2  * 

rr  r2 

V 

rl 

r  3 ' 

rl 

bl' 

r2 

b2' 

r2 

rr  r 3 

r  2 ' 

r  3 

r  3 ' 

r  2 

br 

r3 

V 

r3 

Since  the  first  ball  was  found  to  be  black,  the  equally  likely 
outcomes  for  the  second  ball  being  red  are  only  those  8  among  20 
original  equally  likely  cases  that  have  black  in  the  first  place: 
and  the  favorable  cases  for  the  second  being  red  are  those  among 
the  8  equally  likely  cases  with  red  in  the  second  place.  So,  we 
have 


P (R | B)  *  6/8  =  3/4. 

More  generally,  we  have  the  following  rule  for  calculating  the 
probability  of  dependent  events. 


Rule  4.  If  E  and  F  are  dependent  events  (i.e.,  the  result  of 
event  F  depends  on  the  results  of  an  earlier  trial  of  which 
event  E  is  a  possible  outcome) ,  then 


P(E  and  F)  =  P(E)  x  P(F(E) 

,  P(E  and  F) 

or  inversely  P(F|E)  =  — -  r—r - 

P  vE ) 

2.2.2  Events  not  Mutually  Exclusive:  In  the  example  under 
Rule  two  the  events  were  mutually  exclusive.  That  is  the  success 
of  event  E  "drawing  a  heart"  precluded  event  F  "drawing  the 
Queen  of  Spades.  They  both  couldn't  happen  on  the  same  draw. 
Suppose  the  question  had  been,  what  is  the  probability  of  drawing 


a  heart  or  a  queen  from  the  deck?  In  this  case  the  probability 

11 

52 


of  event  E,  "drawing  a  heart"  is  P^C) 


The  probability  of 


event  F,  "drawing  a  queen"  is  P(F)  --  But  the  probability 

13  +  4  52 

P(E  or  F)  is  not  — -rr —  since  one  event  (the  Queen  of  Hearts)  is 

52 

common  to  both.  The  events  are  not  mutually  exclusive.  To  com¬ 
pute  the  probability  of  "either  a  heart  or  a  queen"  the  ratio  of 
the  number  of  hearts  plus  the  number  of  queens  minus  the  number 
of  queens  of  hearts  is  taken  to  the  total  number  of  cards 


P(E  or  F)  = 


13  hearts  +  4  queens _ -  1  queen  of  hearts  _  J_6 

52  cards  52’ 


Rule  5.  If  E  and  F  are  not  mutually  exclusive,  then  P(E  or  F) 
P (E)  +  P (F)  -  P(E  and  F) . 


2.3  S  UMM ARY 

The  probability  of  a  simple  event  was  defined  in  the  rati-O  of 
successful  outcomes  to  possible  outcomes.  The  probability  of  a 
compound  event  was  shown  to  be  equally  the  ratio  of  successful 
outcomes  to  possible  outcomes  where  the  outcomes  are  described 
in  somewhat  more  complicated  ways.  The  five  rules  given  are 
adequate  to  compute  the  probability  of  any  combination  of 
events,  provided  the  probabilities  of  the  individual  events  can 
be  determined.  To  facilitate  computation  certain  mathematical 
techniques  may  be  employed. 

3 •  BINOMIAL  PROBABILITY  DISTRIBUTION 

3.1  LARGE  NUMBERS  OR  TRIALS : 


Consider  the  following  experiment:  Ten  coins  are  tossed.  What 
is  the  probability  that  exactly  two  of  tfwm  are  heads?  This 
simple  ptoblem  is  complicated  somewhat  by  the  fact  that  the 
number  of  equally  lirtely  outcomes  and  outcomes  favorable  to  an 
event  is  large,  and  enumeration  of  them  j.s  impractical.  For 
example,  if  we  tried  to  list  the  •uually  likely  cases  we  would 
have 

HHHHHHHHHH ,  HHHHHHHHHT,  HHI1HHHHHTH ,  HiniHiniHTHH , 

...,  HHHHHHHHTT ,  HHHHHHHTTT,  - 

and  so  on.  mo  compute  this  probability,  we  can  reason  as  foli- 
F.aeh  coin  has  two  possible  outcomes,  heads  or  tails,  and 


cws  : 
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the  occur'-  either  of  these  to  one  does  not  affect  what  hap¬ 

pens  to  the  others .  Using  the  basic  principle  that  if  event  K  can 
occur  in  at  w.-ys  and  f  in  n.  ways  then  the  event  "E  and  F"  can  occur 
in  m  x  n  ways  (by  rule  3}  we  can  determine  that  there  are  alto¬ 
gether  2x2x2  x  3  x  1  x  2  x  2  x  2  x  2  x  2  -  2 ^  equally  likely 
outcomes . 

I  A 

But  2X  =  1024  and  it  is  no  longer  practical  to  list  these  in 
order  to  look  through  and  pick  out  the  outcomes  favorable  to 
some  event „ 

Looking  at  the  possible  outcomes  we  can  reason  that  there  is  only 
one  of  the  combinations  that  shows  no  heads  (TTTTTTTTTT) .  The 
number  of  combinations  that  yielcfeone  head  are  10,  a  head  in  any 
one  ^f  the  ten  positions.  Going  on  c  s  tep  further,  to  determine 
how  many  ways  two  heads  can  show,  we  know  there  are  ten  ways  one 
head  can  show.  if  one  head  has  shown,  there  are  nine  ways  a 
second  can  show.  The  preduct  of  10  x  9,  then,  gives  us  the  total 
number  of  ways  two  heads  can  show.  Since  we  make  no  distinction 
between  H(1)H(2)  TTT . . . .  and  H(2)H(1)  TTT....  each  successful 
outcome  has  been  counted  twice.  So  the  number  of  combinations 
that  yield  two  heads 

10  10  x  9  10 

(1  i~  ... - — .  The  notation  {  )  is  presently 

Z  1  X  z  z. 

used  in  statistical  work,  replacing  the  symbol  you  probably 

learned 

Following  the  reasoning  to  com put  the  number  of  combinations 
yielding  three  heads,  we  can  soy  that  if  2  heads  have  shown, 
there  arc  8  remaining  coins  leaving  8  ways  the  third  coin  can 
show.  So  there  are  10  x  9  x  8  possible  ways  3  coins  out  of  10 
can  be  heads.  Again  there  are  duplications,  in  this  case  2x3 
or  6.  The  number  of  discrete  combinations  is  .10  _  10x9x8 

1  3  1x2x3' 


Continuing  the  reasoning  we  can  state  that 
1  u  x  9  x  8  x  /  x  b  x  a  x  4 


(L?) 


is  the  number  of  combinations 


Ix2x3x4x5x6x/ 
in  whi~h  7  heads  show.  If  we  compute  the  number  of  combinations 

10 

in  which  3  tails  show,  v.-e  find  (  ) 


10  x  9x8 
1x2x3' 


Since  3  tails 

,  ,  10, 
result  (  ) 


is  the  same  as  7 
10  x  9  x  8  10 

1x2x3  7 


heads,  those  should  give  the  same 
10  x  9  x9x7xftx#x4 
Ix2x3x4x£x$x7’ 


I 
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3.2  DEFINITION 


The  product  of  all  the  integers  from  1  to  n  is  termed  "n  factor¬ 
ial"  or  ni.  Using  this  notation,  the  term  (£?)  can  be  written 
nl 

r  i (n-r )  1 

For  the  term  this  becomes 


10x9x  (8x7  x6x5x4x3x2xl) 
lx2x  (Ix2x3x4x5x6x7  x8) 


45,  noting 


that  the  portions  in  parenthesis  cancel. 

45 

The  probability  of  two  heads  in  ten  coins  is  then  y  -  =  .044. 


3 . 3  BINOMIAL  THEOREM 


This  can  be  reached  in  another  way.  The  number  of  ways  (r) 
successes  and  (n-r)  failures  can  occur  in  n  events  is  (£)  as 
defined  above.  The  probability  that  an  individual  success  (heads) 
will  occur  is  p,  in  this  case  1/2. 

The  probability  that  the  event  will  occur  in  exactly  this  way, 

(r)  successes  (heads)  and  (n-r)  failures  (tails)  is 
(™)  P'  (1  -  P)r‘*'  or  (”)  p‘  qn"  where  q  =  1  -  p. 

This  term,  called  the  binomial  probability  distribution,  is 
mathematically  the  same  as  the  computation  used  previously. 

Tables  (1)  are  available  that  tabulate  the  terms  for  values  of 
n  up  *-o  50  for  any  value  of  p  between  0  and  1  in  increments  of 
.01. 

3.4  BINOMIAL  AS  A  PROBABILITY  DISTRIBUTION 

The  term  probability  distribution  refers  to  the  probability  of 
achieving  various  events. 

The  distribution  of  expected  outcomes  is  proportional  to  the 
probabilities  of  the  individual  outcomes.  For  the  10  coins, 
the  number  of  ways  in  which  favorable  outcomes  can  occur  are: 


Favorable  Outcome 


Ways 


Probability 


0 

Heads 

1 

.001 

1 

Heads 

10 

.010 

2 

Heads 

45 

.044 

3 

Heads 

120 

.117 

4 

Heads 

210 

.205 

5 

Heads 

2  52 

.246 

6 

Heads 

210 

.205 

7 

Heads 

120 

.117 

8 

Heads 

45 

.044 

9 

Heads 

10 

.010 

10 

Heads 

1 

.001 

1024 

1.000 

Plotting  this  on  a  histogram  provides  this  view  of  the  binomial 
probability  distribution  {Figure  4-14'/  . 

4 .  EMPIRICAL  PROBABILITY 

4.1  FUNDAMENTAL  CONCEPT 

The  previous  discussion  of  the  Binomial  Probability  distribution 
illustrated  the  fact  that  the  probability  that  exactly  r  successes 
would  be  observed  in  n  trials  could  be  expressed  as 

P  (X  =  r)  =  (£)  pr  q"*r 

where  q  =  1  -  p  and  p  is  the  constant  probability  of  success  on 
each  trial.  In  previous  discussions,  the  probability,  p,  of 
success  on  each  trial  was  determined  by  deduction.  However, 
this  binomial  distribution  form  can  also  be  used  when  p  is  de¬ 
termined  analytically,  say  by  an  integration,  and  turns  out  to 
be  an  irrational  number.  Even  when  p  is  unknown  -  the  only  re¬ 
quirement  is  that  p  be  constant  for  each  of  the  n  trials.  The 
discussion  of  Empirical  Probability  relates  to  the  problem  of 
what  can  be  done  when  p  is  unknown. 

4.2  DEFINITION 


In  the  events  v*e  have  been  discussing  we  have  been  able  to 
compute  the  probability  of  an  event  by  enumeration.  We  could 
compute  the  number  of  possible  outcomes  and  of  these  identify 
those  we  considered  favorable.  In  a  large  class  cf  events, 
however,  we  cannot  identify  or  count  the  equally  likely  outcomes. 
Can  we  predict  whether  or  not  it  will  rain  on  the  Fourth  of  July. 
There  are  two  possible  outcomes,  rain  or  no  rain,  but  we  cannot 
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say  they  are  equally  likely.  If  we  had  maintained  records  of 
rain  in  Washington  over  the  last  hundred  years  we  could  identify 
the  number  of  times  event  E  "rain  in  Washington  on  the  Fourth  of 
July"  had  occurred.  From  this  we  could  make  an  estimate  of  the 
probability  of  its  occurrence  this  fourth. 

The  probability  given  by  our  previous  definition  is  called  "a_ 
priori , "  or  prior ,  probability  because  the  probability  of  an 
event  can  be  deduced  directly  without  actual  experimentation, 
e  g.,  the  probabilities  associated  with  a  die  are  deduced 
directly  from  observing  the  uniformity  and  symmetry  of  the  die 
and  the  "fairness"  of  its  tosser.  On  the  other  hand,  the  follow¬ 
ing  definition  requires  that  enough  experimentation  can  be  per¬ 
formed  in  order  to  study  the  relative  frequency  of  the  occurrence 
of  an  event;  in  fact,  henceforth,  we  will  confine  our  discussion 
to  probabilities  of  events  that  can  only  be  determined  by  re¬ 
peated  trials  and  the  use  of  this  definition.  Probabilities 
defined  in  this  way  are  called  empirical  or  experimental.  Thus 
we  have  assumed  that  there  is^  a  number  which  gives  the  correct 
probability  of  an  event,  although  one  cannot  say  what  that  num¬ 
ber  is.  Furthermore,  we  will  assume  that  the  empirical  probabil¬ 
ity  obeys  all  the  rules  developed  for  "a  priori"  probability. 

If  whenever  a  series  of  many  trials  is  made,  the  ratio  of  the 
number  of  times  the  event  E  occurred  to  the  total  number  of 
trials  is  nearly  some  constant  p,  and  if  the  ratio  is  usually 
nearer  to  p  when  a  longer  series  of  trials  are  made,  then  we 
agree  in  advance  to  define  the  empirical  probability  of  E  as  p. 

4.3  THE  ROLE  OF  "EMPIRICAL"  PROBABILITY 

Whenever  the  subject  of  tossing  coir.'.s  arises,  everyone  readily 
agrees  that  a  priori  probability  of  heads  is  1/2.  This,  of 
course,  is  because  they  believe  that  the  symmetry  and  uniform¬ 
ness  of  coins  insure  the  equal  likelihood  of  heads  or  tails  — 
which i results  in  the  value  1/2.  As  an  example,  let  us  consider 
tossing  thumbtacks.  When  a  thumbtack  is  thrown,  it  falls  "point 
up"  or  else  "point  down",  and  even  though  it  possesses  symmetries 
and  uniformness,  it  is  very  difficult  (if  not  impossible)  to 
assign  a  priori  probabilities  in  this  case.  To  find  the  pro¬ 
bability  of  "point  up",  one  would  simple  calculate  the  empirical 
probability  of  this  event. 

This  was  tried  experimentally,  a  thumbtack  being  flipped  2750 
times.  Of  these,  the  tack  fell  "point  up"  2054  times.  From 
this  we  can  define  the  empirical  probability  of  "that"  thumbtack 
falling  point  up  on  any  succeeding  trial  or  series  of  trials  as 
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.747.  One  can  then  define  the  Probability  of  3  "points  up"  in 
5  thumbtack  tosses  as  (^)  (.7 47) 3  (.253)2. 

We  will  go  into  greater  depth  on  the  treatment  and  application 
of  empirical  results  when  we  introduce  statistics  several 
lectures  from  now.  The  point  being  made  here  is  that  we  can, 
with  certain  justification,  apply  the  results  of  past  trials  to 
estimating  the  probability  of  success  or  failure  of  future  trials. 

4.4  ASSUMPTION  OF  EQUALLY  LIKELY  EVENTS 

Probability  deals  with  the  prediction  of  future  successes.  Once 
a  trial  has  taken  place  it  becomes  a  statistic.  It  either 
succeeded  or  failed.  Probability  is  no  longer  associated  with 
that  particular  trial.  The  statistic,  representing  an  event  of 
the  past,  does  however  provide  some  information  useful  in  the 
future.  Having  once  conducted  the  trial  we  have  some  assurance 
that  if  exactly  the  same  conditions  are  encountered  on  some 
future  trial,  the  same  results  will  Be  obtained.  It  is  in  the 
inexactness  of  the  repetition  of  conditions  that  probability 
theory  finds  its  place.  In  drawing  from  a  deck  of  cards  if  the  r 

23rd  card  from  the  top  were  drawn,  this  would  be  a  certain  card.  t 

If  the  deck  were  again  ordered  exactly  as  before  and  the  23rd 
card  drawn,  it  would  still  be  the  same  card.  This  is  not  a 
probabilistic  study.  Probability  assumes  that  the  card  is  drawn 
at  random,  that  every  card  has  an  equal  chance  of  being  in  the 
position  selected.  Probability  theory  assumes  that  there  is  a 
certain,  but  not  necessarily  known,  distribution  function  descri¬ 
bing  all  possible  outcomes  of  the  event. 

4.5  VALIDITY  OF  PROBABILITY  THEORY  IN  RELIABILITY  ENGINEERING 


In  the  operation  of  equipment,  failures  occur.  These  failures 
are  caused  by  physical  causes,  such  as  wear,  overstress,  deteri¬ 
oration,  contamination,  etc.  In  some  kinds  of  equipment  these 
failures  can  be  predicted  fairly  precisely.  In  others,  they  seem 
to  occur  randomly,  at  unpredictable  times.  Where  they  are  pre¬ 
dictable,  the  cause  is  soon  known.  Where  they  are  not,  evaluation 
of  the  failures  indicate  many  different  causes.  From  this  we 
draw  the  conclusion  that  in  the  unpredictable  case  a  large  number 
of  factors  are  at  work,  each  causing  some  of  the  failures. 
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5 .  FAILURE  DENSITY  FUNCTIONS 

5.1  EQUIPMENT  FAILURES 

In  the  testing  of  equipment,  and  in  its  operation,  records  have 
been  kept  of  failures  and  operating  times  to  failure.  Histo¬ 
grams  have  been  prepared  showing  the  relationship  between  opera¬ 
ting  times  and  failures.  Among  others,  the  histograms  shown  in 
Figure  4-18seem  to  occur  with  relatively  high  frequency.  A 
histogram  like  this  displays  the  distribution  of  times  to  fail¬ 
ure  and  is  called  a "density  function  ."  Engineers  have  identi¬ 
fied  these  distributions  as  characteristic  density  functions 
which  describe,  to  a  degree,  the  relationship  of  past  failures 
to  the  incidence  of  the  physical  factors  causing  them.  If  this 
interpretation  is  true,  then  the  histograms  shown  may  be  used 
as  probability  density  functions  describing  the  probability  of 
failure  with  time.  Later  we  will  impose  some 

severe  restrictions  on  the  use  of  these  functions,  but  for  now, 
we  will  say  one  can  be  identified. 

5.2  USE  OF  PROBABILISTIC  MAT HEM AT TGS  IN  RELIABILITY 

The  dictionary  defines  reliable  as  trustworthy,  suitable  or  fit 
to  be  relied  on.  Reliability,  then,  is  the  degree  to  which 
equipment  may  be  trusted  to  do  a  job.  Because  of  the  apparent 
relationship  to  probability  tueory  shown  in  testing,  reliability 
has  been  defined  as  the  probability  that  equipment  will  perform 
within  specifications  for  a  specified  time  when  operating  in  a 
specified  environment.  When  we  use  the  term  "within  specifica¬ 
tions,"  included  in  the  meaning  is  " failure- free  operation." 

In  probabilistic  language,  reliability  is  the  ratio  of  the 
subset  of  failure  free  operations  to  the  set  of  all  attempted 
operations.  Probability  theory,  and  its  application  to  estima¬ 
ting  the  probability  that  a  failure  will  occur,  or  will  not 
occur,  within  a  designated  time,  is  one  of  the  major  mathematical 
tools  of  reliability  engineering. 


6.  APPLICATION  TO  RELIABILITY  COMPUTATION 

As  we  discussed  in  the  last  lecture,  systems  are  not  simple 
entities.  They  consist  of  a  great  many  parts,  each  of  which  is 
subjected  to  different  working  conditions,  environments,  stresses 
In  attempting  to  evaluate  the  reliability  of  an  equipment  or 
system  two  approaches  are  available. 


(1)  Build  some  and  test  them,  or 
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(2)  Tost  the  parts  ana  combine  the  probability  of  success 
of  all  the  parts  in  such,  a  way  as  to  determine  the 
probability  of  failure  o f  the  assembled  unit  . 

6-1  COMPLETE  SYSTEMS 


In  the  first  approach,  we  use  the  test  results  to  determine  a 
single  failure  density  function,  the  distribution  of  times  to 
failure.  The  fraction  of  failures  that  occur  prior  to  the  time 
of  interest,  ,  is  used  as  the  probability  that  a  failure  will 

occur  prior  to  time  tt  .  We  call  this  probability  q,  .  The 
number  obtained  by  subtracting  q.  from  1,  (1-q. ),  is  the  pro¬ 

bability  that  no  failure  will  occur  prior  to  time  t{ .  This  we 
have  defined  as  our  reliability. 

6.1.1  Two  Useful  Distributions:  As  mentioned  earlier,  two  dis¬ 
tributions  have  been  found  to  occur  most  frequently.  These  dis¬ 
tributions  have  been  approximated  by  mathematical  functions  use¬ 
ful  in  computation  of  the  reliability.  For  reasons  which  will 
become  clear  i-’  the  statistical  section,  we  normally  predict 
from  the  physical  factors  involved  which  distribution  function 
should  apply,  fitting  our  data  to  the  curve  assumed.  Two  fun¬ 
ctions  useful  for  this  purpose  are: 

(a)  Normal  or  gauss i an 

(b)  Exponential. 

These  are  shown  overlaid  on  the  histograms  of  failure  times  in 
Figure  4-20.  The  curves  show  the  (idealized)  probability  of 
failure  at  any  time.  The  probability  that  the  equipment  will 
have  failed  by  a  rime,  tt  ,  is  the  aggregate  of  the  probabilities 
tha*  it  fails  at  times  prior  to  tt  ,  that  is  the  area  under  the 
curve  from  t  =  0  to  t  =  t, .  Figure 4-21  shows  the  normal  density 
function  and  its  relationship  to  the  reliability  function.  In 
A  the  hatched  area  shows  the  aggregate  probability  of  failure 
to  time  t,  .  This  is  the  value  a,  .  The  value  (.1  -  q,  )  is 
plotted  against  time  in  B.  This  value  is  the  reliability  to 
t ime  t «  . 


6.1.2  The  Normal  (Gaussian)  Function:  This  typifies  the  situ¬ 
ation  caused  by  wearout  of  a  single  part.  It  applies  where  the 
failure  pattern  is  caused  predominantly  by  the  failure  of  one 
particular  part,  as  for  example  in  a  pump  with  a  bearing  greatly 
overloaded  or  of  poor  quality.  The  probability  of  successful 
operation  to  time  t,  is  the  probability  that  that  one  part  will 
operate  to  that  time.  In  this  case,  variations  in  quality  or 
loading  cause  minor  deviations  from  a  characteristic  or  mean 
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life.  Very  few  failures  occur  at  times  greatly  distant  from  that 
mean  life. 

The  characteristics  cf  this  function  are  defined  by  two  para¬ 
meters.  The  mean  life  u  and  the  standard  deviation  - :  u  is  the 
measure  of  central  tendency,  the  average  of  the  recorded  lives; 
a  is  the  measure  of  dispersion  from  that  average. 

Tables  of  the  ordinates  of  the  normal  curve  and  areas  under  the 
normal  curve  with  mean  zero  and  standard  deviation  1  are  pub¬ 
lished  in  most  reliability  books  and  many  other  books  of  mathe¬ 
matical  tables  (2',  .  These  can  be  used  directly  for  reliability 
computations,  entering  with  the  value  Z  =  t...  "  . 

Example:  For  a  part  with  mean  life  u,  -  8  hours  and  standard 
deviation  r  =  2  hours,  find  the  probability  that  the  part  will 

5  —  8 

continue  to  operate  for  5  hours.  Z  =  — =  -1.5.  From  the 

table  of  areas  under  the  normal  curve  for  Z  =  -1.5  we  find  the 
portion  of  total  area  under  the  curve  from  —*■  to  -1.5  to  be 
0668.  This  is  the  probability  that  the  part  will  fail  by  5 
hours.  The  reliability  is  one  minus  this  value  or  0.9332. 

(Figure  4-23) . 

Example:  For  the  same  part,  find  the  probability  that  a  failure 

fi  8  10-8 

will  occur  between  6  and  10  hours  2-  -  — —  =  -1  Z?  =  — —  =  +1. 

The  areas  under  the  curve  are,  for  Zx  -  -1,  area  =  .1507  for 
Z-  =  +1,  area  -  .8413.  The  area  under  the  curve  between  6  and 
10  hours  is  the  difference  or  «. ob26.  The  probability  that  a 
failure  will  occur  during  this  period  is  .6826. 

6.1.3  Exponential  Function :  This  typi f ies  the  " chance" 
failure  rate  function  found  to  be  evident  in  a  large  preponder¬ 
ance  of  situations.  It  appears  to  be  typical  of  most  electronic 
systems  and  numerous  mechanical  systems.  It  indicates  a  bal¬ 
anced  design,  in  effect,  where  no  single  part  (or  few  parts) 
failures  predominate.  The  characteristic  of  the  exponential 
distribution  is  that  the  probability  of  failure  is  constant  for 
any  equal  periods  of  time.  This  constant  probability  is  defined 
by  a  characteristic  mean  time  between  failure,  MTBF.  The 
density  function,  or  distribution  of  times  to  failure  takes  the 
form  of  the  negative  exponential  equation  Xe-^-  where  e  repre¬ 
sents  the  base  of  naperian  or  natural  logarithms  e  -  2.71828 
(Fig.  4-24).  The  reliability  is  again  1  minus  the  area  -|o  time  t, 
so  R  =  where  x  is  the  reciprocal  of  the  MTBF  ( x --  and 

t  is  the  interval  ct  time  of  operation  of  equipment.  The  symbol 
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>  is  defined  as  the  failure  rate.  The  characteristics  of  this 
function  are  oompl  otoly  do  f  i  ned  by  the  one  parameter  ,  MTBF.  If 
the  value  of  this  MTBF  parameter  (or  its  reciprocal,  ,1  is  known 
then  one  can  compute  probabilities  of  failure  occurrences  during 
qiven  intervals. 

Example:  The  MTBF  for  a  part  is  100  hours.  Find  the  probabil¬ 

ity  that  the  part  will  fail  in  the  first  10  hours  of  operation. 


P(t  *  10) 


f  10 


o 


100 


-t /100 

e  d  t  •- 


-10  100 

e 


1  -  e"U‘  1  1  -  0.90  5  -  0.09  5 


The  reliability  of  operation  of  this  part  over  the  10-hour 

,  .  -10/100  n 
interval  is  e  -  0.905. 


Example:  The  reliability  of  a  part  is  known  to  be  0.98  over  a 

150-hour  time  of  operation.  Find  its  failure  rate. 


0.98 


-)  x  150 


In  0.98 


-  x  150 


-  lnO-98 
1  50 


.02020 

150 


=  .000135 


Example:  A  part  has  ?n  MTBF  of  100  hours.  Compute  the  probabil¬ 

ity  that:  this  part  will  not  fail  in  the  time  interval  [100  hours 
to  200  hours  1  qiven  that  it  operated  successfully  for  the  first 
100  hours . 


If  we  let  A  symbolize  the  event  "no  failure  in  the  interval 
f  100,  200  i"  and  B  the  event  no  failure  in  0,  100 j,  the  desired 
probability  is  seen  to  be  a  conditional  probability,  P(A'B). 
According  to  the  definition 


p(a'b)  ■ 


P ( A  and  B) 
P(B) 


Now,  A  and  B  is  the  event,  "no  failure  in  the  interval,  ?0,  200)' 

,  ,  ,  -200/100  -2 
F ( A  and  B)  =  e  =  e 


P  (B) 
P(AlB) 


-100/*00  -1 
=  e  -  e 


-2  .  -1 
=  e  /  e 


-1 


0.368 
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One  point  that  should  be  noted  is  that  the  reliability  tor  a 
period  of  time  equal  to  the  MPBF  is  only  . 3? .  A  second  point  is 
also  of  interest  --  given  100  hours  of  successful  operation,  the 
probabil ity  of  10n  future  lours  of  successful  operation  can  be 
computed  on  the  same  basis  as  100  hours  on  new  equipment.  The 
same  statement  can  be  made  if  we  consider  an  interval  of  .100 
hours  after  ar,n  hours  of  successful  operation.  In  particular, 
for  the  negative  exponential  case  results  from  tests  involving 
successfully  used  equipment  rather  than  new  equipment  are  still 
valid . 


This  type  of  result  is  unique  to  the  negative;  exponential  and 
is  not  applicable  for  Normal,  We ibu  1 .1 .  or  other  distributions 
of  times  to  failure  where  the  failure  rate  is  not  constant . 


6.2 


PREDICTING  RELIABILITY  OF  SYSTEMS  FROM  RELIABILITY  OF  PARTS 


6.2.1  Reasons  for  1  redaction;  Building  equipment  arid  testing 
it  to  determine  reliability  provides  this  result  too  late  to 
influence  the  design.  A  designer  would  prefer  to  evaluate  the 
reliability  of  the  design  before  he  spends  money  on  production 


This  would  permit  him  to  make  changes  to  his  proposed  design  to 
meet  the  requirements  at  greatest  economy .  The  alternative  to 
testing  the  complete  unit  is  dett  mining  the  reliability  charac¬ 
teristics  of  the  parts  that  make  up  the  unit  and  then,  somehow, 
combine  this  information  to  ascertain,  the  reliability  of  the 
assembled  unit.  This  is  the  probability-  of  a  compound  event, 
discussed  earlier.  The  five  rules  presented  at  that  time  are 
the  rules  of  combination  of  simple  events  to  determine  the  pro¬ 
bability  of  a  compound  e\ent.  The  compound  o*-ent  is  the  relia¬ 


bility  or  probability  of  success  tor  a  certain  t  imp  of  toe 
equipment .  The  simple  events  are  the  reliabilities  of  the 
individual  parts. 


6.2.2  Case  1,  series:  If  the  successful  operation 
part  is  necessary  for  the  success  of.  the  equipment .  t 
plication  rule  applies.  The  probabi  !.ity  of  the  sueees 
of  all  of  a  series  or  irony  of  events  is  the  product 
individual  probabilities  os  success.  This  is  an  appl 
our  probability  combination  rule.  Rule  1. 


all  the 


3  fa  1  outcome 
o  i  a  1 1  t  h  e 
i cat  ion  c-t 


R.  x  K..  X  R 
i  1 


R  ,  x  iX ,  ■  x  ...  x  R 


['his  is  like  a  chain.  If  any  link  breaks ,  the  chain  fails. 


6.2.1  Case  2,  Fur  a  lie  I:  If  the  equipment  will  succeed  if 
either  of  two  (or  more)  parts  perform  successfully,  then  it  will 
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fail  only  if  both  (or  all)  parts  fail.  Using  the  multiplication 
rule  (Rule-  1)  with  rule  1. 

=  1  -  ( 1-R . )  x  (1-RJ  x  ...  x  ( 1-R  ) 
u  i  Z  N 

6.2.4  Case  3,  Series-Parallel:  It  is  frequently  the  case  that 
both  series  and  parallel  situations  exist  simultaneously  in  the 
equipment.  To  determine  the  reliability  of  the  equipment,  re¬ 
solve  each  parallel  situation  into  its  equivalent  reliability 
then  the  total  may  be  treated  as  a  series  case. 


SAMi  LE  COMPUTATIONS 


1  SERIES  CASE  -  NORMAL  DISTRIBUTION 


Normal  or  gauss ian  failure  density  functions.  Assume  an  equip¬ 
ment  E  consisting  ot  four  parts  (A,B,C,D)  (Figure  -.-281,  in  series 
with  the  following  characteristics. 
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;;se,  assumes  that  ail  parts  are  new,  starting  from 
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time  0. 

7.2  GENERAL  SERIES  CASE  -  NORMAL  DISTRIBUTION 

In  the  mere  general  case,  the  parts  have  various  operating  times 
already  accrued,  as  for  example  when  the  equipment  is  operated 
to  failure,  repaired  and  continued  in  operation.  We  might  look 
at  the  reliability  of  the  equipment  for  some  other  two  hour 
period,  say,  from  the  14th  to  16th  hours  {Figure  4-31) . 

Wht  e  this  situation  obtains,  the  normal  curves  have  to  be  re¬ 
drawn  to  start  each  at  the  time  the  repaired  part  started  opera¬ 
ting,  still  with  the  same  mean  life  and  same  standard  deviation. 
The  reliability  for  the  two  ,our  period  in  question  is  still  the 
product  of  individual  reliabilities.  The  probability  that  any 
part  will  fail  during  a  particular  period  is  the  area  under  the 
normal  curve  for  the  period  in  question,  obtained  by  subtracting 
the.  area  under  the  tail  of  the  curve  {-»  to  t2  )  . 

Knowing  that  the  equipment  has  not.  failed  at  time  t  =  14, 
implies  each  part  has  a  conditional  probability  of  failure, 
given  that  it  has  not  failed  at  time  t  =  14.  The  conditional 
probability  that  the  equipment  will  fail  during  the  period  14 
hours  to  16  hoc  s  is  the  ratio  of  the  area  under  the  individual 
curves  between  14  and  16  hours  to  the  area  under  the  curve  from 
14  hc-urs  to  +  ®.  In  the  case  shown,  the  reliability  of  the 
equipment  is  found  to  be. 

SERIES  CASE  -  NORMAL  DISTRIBUTION 
TO  FIND  RELIABILITY  OF  EQUIPMENT  E  FROM  TIME  t  =  14  to  t  =  16 


PART 

A 

B 

C 

D 

1 

time  of  last  failure, 
hours  from  0 

12.6 

10.8 

11.4 

9.6 

2 

mean  life  measured 
from  last  failure 

16.6 

15.8 

17.4 

19.6 

3 

Z  =  — — ^  computed  for 
'  14  hours 

-2  .6 

-1,34 

-2.27 

-2.24 

4 

Z  =  — computed  for 
°  16  hour3 

-1.6 

+  .16 

-.93 

-1.44 

5 

Area  under  normal  curve 
(~®  to  16  hours) 

.0  548 

.5636 

.1762 

.  0749 

6 

Area  under  normal  curve 
(-®  to  14  hours) 

.0047 

.0901 

.0116 

.0125 
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PART 

A 

B 

c 

n 

7 

Difference 

.0501 

-  47  3  5 

.1646 

.0624 

8 

R 

.9499 

.  5265 

.8354 

. r  37  6 

9 

re 

.3917 

SERIES  CASE  -  EXPONENTIAL  DISTRIBUTION 


When  the  negative  exponential  distribution  applies  to  the  parts 
failure  rate?,  the  probability  of  failure  is  independent  of 
prior  operation.  For  any  period  of  time  t  the  reliability  of 
the  equipment  for  the  serial  case  is  the  product  of  the  relia¬ 
bilities  of  the  parts,  that  is 

_x  t  -X  t  -X  t  X  t 

R„~e  xe  xa  x  ... 

E 


=  e 


-<XA  +  Xt. 


>  xE  ...It 


The  reliability  of  an  equipment  consisting  of  four  serial  parts 
with  KTBFa ,  as  before  of  4,  5,  6  and  1C  hours,  for  a  2-hour 
period,  can  be  computed  as  follows; 


Part 

MTBF 

\  .  1 

MTBF 

R  =  e~ 

A 

4 

.25 

.606 

B 

5 

.20 

.670 

C 

S 

.183 

.691 

D 

10 

.100 

.818 

.733 

K  “  e~ 

<!') 1  =  e-733 

x  2  -1.467 

=  e 

-  .230 

7.4  series  parallel  case  -  exponential  distribution 


In  the  previous  examples,  it  was  assumed  that  the  failure  of  any 
one  component  would  cause  equipment  failure.  If  there  are 
parallel  components  or  series  parallel  combinations  the  principles 
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or  coiuL.1  ud lion  ol  prolm'uiiities  discussed  earlier  apply.  A  com- 
putation  will  be  performed  for  such  a  combination  to  explain  the 
procedure.  An  equipment  consists  of  four  parts,  (Figure  4-34) 
with  part  B  such  that  a  failure  of  3,  would  not  constitute  an 
equipment  failure  unless  B?also  failed.  Likewise  a  failure  of 
Eo  would  not  constitute  equipment  failure  unless  Bx  also  failed. 

We  assume  a  negative  exponential  distribution  of  times  to  fail¬ 
ure  for  all  parts.  The  reliability  of  the  equipment  is  the 
product  of  the  reliabilities  of  each  part  A,  B,  and  ,  C  and  D, 
where  3-  and  B;  must  be  considered  together. 


MTBF 

> 

R  ( 2-hou] 

A 

20 

.05 

.905 

Bi 

10 

.  10 

.819 

B; 

15 

.067 

.875 

C 

25 

.04 

.923 

D 

3  0 

.03  3 

.939 

But  the  probability  that  both  Si  and  Be  fail  is  the  product 
(1  -  Rp_  )  x  (1  -  kg,,  )  .  The  combined  reliability  is  1  minus  the 
product.  Expanding  the  product  aives  kg,  g.  =  1  -  (1  -  Rg  ) 

( 1  -  %  ) 

-  1  -  1  +  RBl  +  Rfig  ~  Rb^Bb 
=  rB-  r  rBe  "  RB,  rBb 


Rb  =  .319 
R  g  -  .875 


Rrj  X  R 


B. 


=  .7  16 


RblB?  =  *978 

The  reliability  R  =  R  xRxHx.R 
1  E  B)  Be  A  C  D 


These  last  three  terms  may  be  combined  as  before  to 
R _ =  e 


+  +  X)t 

A  C  D 


ACD 


C<-123)2  -  .781 


then  R_  =  (.978)  x  (.781)  =  .7S2 
E 
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7  ♦  5  STAiSTDBY  PARALLEL  CASE  -  EXPONENTIAL  DISTRIBUTION 

The  foregoing  example  assumed  both  B1  and  operating  simultan¬ 
eously  as,  for  example,  two  generators  operating  in  parallel, 
where  it  is  clear  that  one  can  handle  the  entire  load  without 
increasing  the  probability  of  its  failure.  Let  us  look  at  a 
slightly  different  situation,  (Figure  4-36),  two  generators,  one 
operating  the  other  not  operating  unless  the  first  fails,  at 
which  time  it  will  be  substituted  for  the  first.  Two  switches, 

S,  and  SP  have  been  added  to  indicate  the  increased  complexity 
of  the  system. 

The  switch,  S,  or  Sp  ,  can  fail  in  either  of  two  ways.  Considering 
Si  s 

(1)  It  can  fail  to  cause  a  transfer  from  to  B2  when  Bj 
fails ; 

(2)  It  can,  in  error,  cause  a  transfer  to  Bp  when  B-^  has  not 
failed . 

Let's  take  a  closer  look  at  that  switch.  It  might  be  a  starting 
valve  for  the  diesel  engine  driving  generator  Bg  ,  held  in  the 
closed  position  against  spring  pressure  by  a  solenoid  energized 
whenever  there  is  a  voltage  output  from  Bt  .  The  valve  is,  of 
course,  locked  closed  electrically  whenever  the  generator  it 
starts  is  running.  Failure  by  the  first  mode  might  be  a  mechan¬ 
ical  failure,  such  as  the  valve  freezing  shut  or  a  mechanical 
linkage  broken,  should  this  type  of  failure  have  occurred,  the 
valve  cannot  operate  on  a  failure  of  B:  ,  hence  a  failure  of  Bi 
would  cause  the  entire  system  to  fail.  Failure  of  the  switch 
by  mode  2  might  be  an  electrical  failure,  an  opening  of  the  coil 
permitting  the  valve  to  open,  starting  generator  Bp  and  connec¬ 
ting  it  to  the  line.  An  interlock  is  presumably  provided  to 
shut  down  the  dieSel  engine  driving  generator  B-i  ,  disconnecting 
the  generator  from  the  line,  where  it  remains  ready  to  start 
again  should  generator  Bp  fail.  Should  the  switch  fail  in  the 
second  mode,  the  operation  is  still  successful  unless  the  gener¬ 
ator  Bp  fails.  If  this  should  occur,  the  identi^l  switching 
arrangement  on  generator  Bp  cannot  successfully  transfer  back  to 
B!  since  each  attempt  to  transfer  back  will  result  in  the  switch 
failure  trying  to  start  Bp  and  dropping  B-,  off  the  line. 

Looking  at  just  the  four  equipments  (B-,  ,  B?  ,  S1  and  S?  )  we  can 
identify  the  possible  events  that  may  occur.  To  describe  the 
possible  outcomes  an  abbreviated  notation  will  be  used.  § 


| 
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B!  Means  generator  Ba  operates  successfully. 

Bj  Means  generator  B:  fails. 

SMa  Means  switch  Sa  operates  successfully  to  transfer  from 
Bj  to  Ba  • 

SEX  Means  switch  Sa  does  not  fail  electrically  and  so  does 
not  cause  an  unnecessary  transfer  from  to  Bg. 

SMt^  Means  switch  fails  to  activate  to  cause  a  transfer 
to  B2  on  the  failure  of  B5. 

SEX  Means  switch  S:  transfers  the  load  to  Bs  while  Bx is 
still  operating  correctly. 

The  same  notation  will  be  used  referring  to  the  performance  of 
generator  Ba  and  switch  Sa .  (Figure  4-38)  .  Note  that  the  bar 
above  the  abbreviation  denotes  unsuccessful  operation. 

The  probability  that  at  least  one  of  the  eight  indicated 
sequences  occurs  is  1.0.  The  sum  of  the  probabilities  of  all 
possible  sequences,  then  must  idd  up  to  1.0.  To  compute  the 
reliability,  or  probability  of  success,  we  can  compute  the 
individual  probabilities  that  each  successful  sequence  occurs 
and  add  them,  or  we  can  compute  the  probability  that  each 
unsuccessful  sequence  occurs  and  subtract  the  sum  from  1. 

Successful  Sequence  1  can  be  seen  to  bes  BT  operates  success¬ 
fully,  Si  does  not  fail  electrically. 

Successful  Sequence  2  is  seen  to  be:  B*  operates  successfully 
but  Switch  ST  fails  electrically,  transferring  the  load  to  Ba . 
B;j  operates  successfully  and  S8  does  not  fail  electrically. 

Successful  Sequence  3  can  be  seen  to  be:  Bj  fails,  Switch  Sx 
operates  mechanically  starting  generator  Ba .  Ba  operates 
successfully  with  no  electrical  failure  of  S8 . 

Consider  now  the  question.  What  is  the  probability  that  either 
or  Ba  will  operate  successfully  for  a  time  t. 

_(XB  +  XSE.H 

R  (Sequence  1)  *  R  x  R  *  e  1 
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R  (Sequence  2 )  -  R„  x  x  Rnr  x  (1  -  RgE  ) 


=  e 


B, 

-•v  t 


SE- 


-(l  +  >  )—  -1  t 

B-,  SE?  2  .  SE,  . 

x  e  '  ( 1  -  e  ) 


R  (Sequence  3)  =  R  x  R  x  R  x  (1  -  R  ) 

_  +  i  x  >  )  (-)  t 

SM,  By  SE-,  v2  , ,  B,  , 

=  e  (1  -  e  *  ) 

Although  this  is  not  capable  of  reduction  to  a  form  R  =  e 
i  v.  can  be  solved  to  provide  a  numerical  answer  R 


-Xt 


BS 


Tms  answer 


can  then  be  multiplied  with  the  combined  product  R  to  obtain 

1  ACD 

the  final  answer. 

Assuming  a  negative  exponential  distribution  of  times  to  failure 
for  the  various  components  as  in  the  previous  example,  with 
reliabilities  of  the  switches  as  follows: 


MTBF 


pe 


SE1  , 

SE2 

50 

.020 

SMI  , 

SM2 

75 

.013 

probability  of 

the 

succes 

;s  ful 

sequences 

can 

1) 

.905  x 

.961 

- 

.870 

2) 

.90a  x 

.039 

x 

.935  x 

.980 

=  .032 

3) 

.095  x 

.987 

X 

.935  x 

.980 

=  .086 

The  probability  of  successful  operation  for  a  two  hour  period, 
can  be  seen  to  be  the  sum  of  the  probabilities  or  -988. 

The  probability  of  successful  operation  of  the  system  is 
.988  x  .781  =  .763 


. W1 
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Chapter  5 

RELIABILITY  PREDICTION 

Reliability  prediction,  performed  as  part  of  the  system 
development,  is  analogous  to  the  analyses  the  designer  makes  on 
measurable  performance  characteristics  such  as  voltage,  pressure 
or  temperature.  In  design,  the  designer  computes  the  expected 
performance  values,  lie  used  techniques  verified  by  previous 
experience.  Later  testing  merely  confirms  his  analysis.  While 
reliability  is  not  measurable  in  the  same  sense,  it  is  a  tangible 
characteristic  of  the  design.  Reliability  prediction  is  the 
analytical  method  of  determining  w1- at.  the  consequences  of  design 
decisions  made  before  the  manufacture  and  test  of  tne  equipment 
will  be.  Tt  is  based  on  techniques  confirmed  by  previous  ex¬ 
perience.  It  provides  a  quantitative  measure  of  the  reliability 
of  the  equipment  as  designed  which  may  be  compared  to  the  require¬ 
ments  to  assure  that  the  final  design  has  achieved  those 
requirements.  The  operational  reliability  of  the  equipment  may 
be  compared  against  the  predicted  value  to  identify  areas  where 
improved  training  or  improved  production  processes  can  be 
effectively  used.  In  this  chapter  we  will  describe  hew  the 
prediction  is  performed. 

As  described  in  chapter  3,  the  prerequisite  for  performing  a 
system  analysis  is  a  description  of  the  system.  This  description 
is  usefully  provided  by  a  system  model,  including 

(a)  Identification  of  the  system  to  its  component 
parts  . 

(b)  Definition  of  failure  of  the  system  in  terms  of 
functions  required. 

(c)  Environment  in  which  the  system  must  operate. 

(d)  Time  of  required  operation. 

For  reliability  prediction,  the  system  model  must  describe  the 
relationship  between  component  failures  and  system  failures. 

This  relationship  for  most  systems  can  be  adequately  represented 
by  either  series  (serial)  or  parallel,  (redundant)  models  or  by 
combinations  of  the  tw<o  as  discussed  in  Chapter  4.  In  the  series 
system  it  is  assumed  that  a  failure  of  any  of  the  components  will 
result  in  the  failure  of  the  system,  or  in  other  words,  the 
system  will  operate  successfully  only  if  all  the  components 
operate  successfully.  This  is  analogous  to  a  series  electrical 


"  1 


1 ighi ing  circuit .  In  a  parallel  system,  it  is  as.v.;meu  that  the 
svsterr.  will  fail  or.lv  if  all  the  c  rr.eoncnts  fail,  or  in  c-t- nor 
w  ords  ,  the  sy  stent  will  operate  success  fully  if  any  one  of  the 
components  operate  successfully.  This  is  analogous  to  a  parallel 
electrical  lighting  circuit. 

As  a  quick  review,  the  system  models  can  be  mere  precisely 
described  in  terms  of  events,  the  s  ..cress  or  failure  or.  the 
components  of  the  system.  Let  S  denote  the  event  that  the  system 
is  successful  and  S-^  the  event  that  the  jtn  component  operates 
successfully.  The  event  S  in  a  cries  system  made  up  of  m 
components  can  be  expressed  as  a  combination  of  the  events 
S j  ,  j  =  i  ,  2  ,  .  .  .  ,m,  as 

S  =  S  ]_  and  S2  and...  and 

the  intersection  of  the  events  Sj .  In  a  similar  way,  the  event 
S  in  a  parallel  system  can  be  expressed  as  a  combination  of 
the  events  S^. 

J 

S  =  Sp  or  S2  or... or  Sn 
the  union  of  the  S j . 

The  reliabil't.y  of  the  series  system  can  then  be  written 

n  r. 

R  =  P(S)  =  .  n  P  (SJ  =  .  II,  R  . 

1-1  3  3=1  3 

in  which  R3  is  the  reliability  of  the  j^h  component.  In  the 
same  manner,  the  reliability  of  the  parallel  system  becomes 

m  m 

r  -  p ( s )  -  1  -  n  ( i-p (S-; ) )  -  i  -  n  ( 1 — r  . ) 

j=i  j  j=i  J 

Techniques  useful  in  the  analysis  and  prediction  of  equipment 
reliability  have  been  unde  development  since  about  1957.  In 
the  field  of  electronics  the  techniques  have  been  developed 
extensively;  the  methods  utilized  in  evaluating  reliability  in 
mechanical  systems  has  been  a  inoie  recent  development.  At  the 
same  time  that  the  prediction  techniques  have  been  evolving, 
emphasis  has  been  placed  on  the  gather  inn  of  failure-rate  data 
on  parts  and  the  measurement  of  reliability  of  existing  equip¬ 
ments  in  order  to  provide  numerical  significance  to  the  various 
mathematical  expressions  used  in  describing  reliability.  It 
mus t  be  remembered  that  the  real  value  of  these  numerical 
■expressions  lie  m > t  in  th e  number  itself,  but  in  the  information 
it  conveys  and  the  use  made  of  that  in  forma t _i cm .  Reliability 
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predictions  do  not,  in  themselves,  contribute  to  the  reliability 
of  a  system.  Control  cf  failure  frequency  for  any  system  can  be 
improved  with  more  complete  knowledge  of  failure  modes  and  fail¬ 
ure  mechanisms  (Chapter  12) .  Reliability  predictions  provide  a 
set  of  criteria  for  selecting  courses  of  action  for  this  investi¬ 
gation  and,  therefore,  affect  the  actual  reliability  of  a  system. 

Reliability  prediction  techniques  are  those  methods  used  to 
obtain  a  numerical  indication  of  the  inherent  reliability  of  a 
device.  Inherent  reliability  is  the  reliability  potential  of  the 
design,  excluding  the  degradation  which  will  occur  in  production, 
storage  and  operational  use. 

1.  STAGES  OF  DESIGN  vs  PREDICTION 


Reliability  predictions  should  be  started  as  soon  as  the  design 
begins  to  take  shape  in  identifiable  components,  before  the 
selection  of  such  components  as  parts  of  the  system  is  made. 

The  prediction  should  be  used  as  a  working  design  tool,  used  to 
compare  the  effect  of  alternate  possible  courses  of  action  so 
that  the  best  can  be  selected.  The  prediction  should  be  kept 
current  as  the  design  becomes  more  fixed,  to  be  used  in  evalua¬ 
tion  of  interface  problems,  to  confirm  previous  analyses  as  test 
results  accrue  and  to  provide  an  analytical  evaluation  for 
proposed  changes.  The  prediction  changes  as  the  design  develops 
os  follows: 

1.1  PRE-DESIGN 


r edict ions  made  in  the  pre-design  stage  are  based  on  little  or 
no  detailed  design  information.  They  are  used  in  feasibility 
studies,  evaluation  and  comparison  of  alternate  design  configura¬ 
tions,  and  in  reliability  allocation.  Because  of  the  limited 
information  on  which  they  are  based,  these  predictions  cannot  be 
us  precise  as  later  prediction1"  .  These  initial  (pre-design) 
predictions  do  not  contribute  appreciably  to  identifying 
specific  reliability  problems  or  indicating  areas  of  data  de¬ 
ficiency.  However,  by  influencing  decisions  on  design  concepts 
and  the  scope  of  the  reliabi 1 ity  program,  they  can  have  a 
s-.', bat. in'-  iul  influence  on  system  reliability  and  system  develop¬ 
ment  . 


1.2  DESIGN 


Pre-design  predictions  must  be  updated  periodically  in  order  to 
•>id.  in  making  timely  decisions  on  design  details  as  well  as  on 
♦-he  other  elements  of  the  program.  Predictions  during  the  design 
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phase  are  made  after  the  pre-design  prediction  and  prior  to 
design  completion.  As  the  design  progresses,  consecutive 
predictions  can  be  made  with  increasing  precision.  These 
successive  predictions  will  be  based  on  the  accumulated  knowledge 
of  the  parts  to  be  used,  the  application  stresses,  the  manner  irt 
which  the  functions  are  accomplished,  and  the  environmental 
conditions  to  which  the  parts  will  be  subjected. 

1.3  COMPLETED  DESIGN 

When  a  design  is  completed,  an  updated  prediction  is  made.  This 
prediction  will  be  the  best  reliability  estimate,  before  actual 
reliability  measurement  through  operational  testing,  because  it 
will  reflect  complete  design  information.  This  is  not  the  final 
prediction  to  be  made  on  this  design,  however.  As  the  design  is 
changed  the  reliability  prediction  must  be  revised  accordingly. 

2 •  RELIABILITY  PREDICTION  APPROACH 

2.1  TYPES  OF  FAILURE 


A  system  is  a  collection  of  parts  mechanically  and/or  electri¬ 
cally  joined  together  in  order  to  perform  certain  specified 
functions.  If  a  system  is  capable  of  satisfactorily  performing 
its  functions  at  some  point  of  time,  it  will  continue  to  have 
that  capability  until  a  significant  change  occurs  in  the  opera¬ 
ting  characteristics  of  some  part,  or  group  of  parts.  Part 
failure  occurs  when  the  characteristics  of  a  part,  or  group  of 
parts,  have  changed  to  the  point  where  they  exceed  the  limits 
within  which  the  system  functions  are  satisfactorily  performed. 
Whenever  a  system  fails,  a  group  of  parts  have  failed.  Thus, 
the  reliability  of  a  system  is  directly  related  to  the  number  of 
parts  it  contains  and  the  reliabilities  of  these  individual  parts. 

The  prediction  of  the  reliability  of  a  system  is  the  determination 
of  the  expected  reliabilities  of  individual  parts  as  they  are  used 
in  the  system.  The  reliability  of  a  part  is  determined  by  three 
factors:  (a)  characteristics  of  the  part  at  the  beginning  of 

the  operating  period  of  interest,  (b)  the  characteristic  limits 
which  constitute  failure,  and  (c)  the  magnitude  of  the  changes 
occurring  in  characteristics  during  the  period  of  operation, 
which  may  be  directly  related  to  environment,  or  physical  or 
electrical  stress. 

We  consider  two  categories  of  parts  failure.  The  first 
(Catastrophic  Failure)  is  that  in  which  functional  character¬ 
istics  change  abruptly  and  drastically,  e.g.,  a  tube  becoming 
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inoperative  due  to  heater  opening  or  a  pump  bearing  seizure. 

The  second  category  (Drift  Failure)  is  that  in  which  there  is  a 
relatively  gradual  change  in  measurable  functional  character¬ 
istics  until  operation  is  no  longer  satisfactory,  e.g.,  a  tub^ 
whose  transconductance  diminishes  to  the  point  of  failure  due  to 
a  build-up  of  interface  resistance,  or  the  gradual  wear  on  the 
pivot  of  a  cam,  permitting  a  misalignment  in  an  operating 
mechanism . 

Prediction  of  failure  is  a  process  that  remains  basically  the 
same  regardless  of  the  data  or  procedures  used.  It  is  based  on 
the  premise  that  like  parts  have  approximately  the  same  relia¬ 
bility  in  one  system  as  in  any  other  system,  if  they  are 
subjected  to  the  same  stresses.  This  permits  the  application  of 
data  obtained  from  prior  operation  of  parts  to  predict  their 
reliabilities  in  new  systems. 

At  this  point  we  must  further  clarifv  the  relationship  between 
part  failure  and  system  failure.  A  part  used  in  a  redundant 
element  of  a  system  cannot  cause  the  system  to  fail  unless  the 
other  redundant  element  "fails."  For  parts  in  a  redundant 
element,  failure  is  redefined  as  occurring  when  the  character¬ 
istics  of  a  part,  or  group  of  parts,  exceed  the  limits  within 
which  the  system’s  functions  would  be  satisfactorily  performed 
if  the  part(s)  were  not  in  a  redundant  path.  Therefore,  the 
reliability  of  a  system  which  contain''  redundant  elements  is  not 
simply  the  product  of  the  reliabilities  01  its  parts.  A  more 
complete  formula  relating  system  reliability  to  part  reliabili¬ 
ties  must  be  used  for  predicting  the  reliability  of  a  system 
which  includes  redundancy  (see  Chapter  4) . 

2.2  VALIDITY  OF  THE  EXPONENTIAL  DISTRIBUTION 


The  assumption  of  the  exponential  distribution  of  times  to 
failures  in  conducting  predictions  of  reliability  of  systems  is 
usually  made  because  of  three  facts. 

(a)  In  general,  sufficient  data  is  not  available  to 
provide  confidence  in  the  selection  of  an  alternate 
distribut ion. 

(b)  The  mathematical  computation  is  greatly  simplified 
by  this  assumption. 


(c)  It  provides  answers  on  the  conservative  side. 


Lacking  any  theoretical  basis  for  assigning  a  distribution  of 
time  to  failure,  one  naturally  turns  to  empirical  data  for 
possible  generalizations  about  the  nature  of  the  distribution. 

In  1952,  Davis  (3)  published  an  article  containing  an  aruilysis. 
of  failure  data  from  a  wide  assortment  of  unrelated  systems.  He 
concluded,  "The  exponential  theory  of  failure  appears  to  describe 
most  of  the  systems  examined  here.  Those  systems  which  exhibit 
reasonable  agreement  with  this  failure  theory  are  characterized 
by  predominance  of  human  errors  as  the  cause  or  a  careful  and 
well  developed  operating  technique  for  minimizing  failure. 

Systems  which  are  subject  to  a  wide  range  of  environmental 
severity  also  appear  to  follow  this  pattern."  He  also  found 
that  some  of  the  systems  examined  generated  failures  in  a  way 
best  described  by  a  normal  distribution,  but  these  systems  were 
characterized  by  what  we  now  refer  to  as  wearout  failures. 

The  Davis  article  has  been  referred  to  quite  often  as  justifica¬ 
tion  for  the  assumption  of  an  exponential  distribution  of  the  time 
to  failure  of  electron  tubes.  Further  evidence  was  published  in 
a  series  of  ARINC  monographs  (4,  5, .6,  7),  in  which  a  large 
number  of  electron  tube  failures  were  found  to  fellow  an 
exponential  distribution.  Other  electronic  components,  however, 
were  found  to  fail  in  a  manner  best  described  bv  a  normal 
distribution.  Kao  (7)  has  more  recently  found  that  a  Weibull 
distribution  best  fitted  the  failure  data  relative  to  over  two 
thousand  electronic  tube  failures.  Weaver  and  Smith  (8)  have 
found  that  the  failure  times  of  certain  electromechanical  devices 
can  best  be  fitted  by  a  mixed  Weibull  distribution.  The  above 
evidence  casts  doubt  on  the  notion  that  a  single  distribution 
can  safely  be  used  to  represent  all  types  of  failures. 

MacFarlane  and  Mickel  (9)  show  that 'the  exponential  time  to 
failure  assumption  provides  a  reasonably  accurate  solution  in 
the  case  of  normally  distributed  times  to  failures  where  each.  "  jl 
failure  is  repaired  as  it  fails,  as  long  as  the  standard  devia-  j 
tion  is  greater  than  one  tenth  of  the  mean  life  of  the  parts. 

The  important  concept  is  (Figure  5-8)  that  when  a  population  of 
life  parts  enter  service  together  at  time  t  -  o,  they  will  all 
fail  in  a  greater  or  less  concentrated  period  centering  about 
their  mean  life.  When,  however,  each  of  the  part  population  has 
been  replaced  several  times,  thereby  maintaining  the  population, 
the  individual  ages  of  the  replacements  become  so  well  mixed 
that  failures  and  renewals  occur  in  nearly  random  fashion.  As..  - 
the  equipment  becomes  more  complex  and  as  parts  each  portraying 
individual  failure  characteristics  increase  in  number,  the 
"Random"  approximation  improves  in  accuracy. 


9 


& 


I 


The  best  procedure  to  follow  in  selecting  the  distribution  to 
use  in  the  computation  is  to  examine  whatever  evidence  might  be 
at  hand,  select  a  distribution  which  seems  to  be  compatible  with 
the  evidence,  and  submit  the  selected  distribution  to  suitable 
statistical  tests  of  goodness  of  fit.  For  example,  if  it  seemed 
reasonable  to  the  engineer  that  a  constant  failure  rate  would  be 
a  characteristic  of  the  device,  one  may  hypothesize  +hat  the 
time  to  failure  follows  an  exponential  distribution.  If  wear 
out  failures  are  expected,  a  normal  discribution  may  be  a 
suitable  first  estimate.  Experience  with  similar  devices  may 
indicare  t^at  a  W^ibuil  distribution  is  appropriate.  In  the 
absence  of  ary  technical  reasons  for  selecting  a  particular 
distribution,  one  might  examine  -everal  of  the  distributions 
which  may  be  compatible  with  t  ie  expected  failure  pattern.  The 
Weibull,  exponential,  normal,  and  gamma  distributions  would  all 
be  suitable  candidates  for  a  first  approximation.  Distributions 
of  time  to  tail-  re  are  carried  to  greater  depth  in  Chapter  9. 

For  the  purposes  oi  prediction  of  reliability,  in  the  absence  of 
goon  information  on  the  distribution  of  times  to  failure,  the 
assumption  of  the  negative  exponential  distribution  should 
be  useri, 

2.3  RELIABILITY  PREDICTION  APPROACH 

To  accomplish  the  prediction  of  reliability  of  a  complex  system, 
the  fol' owing  steps  are  recommended. 

(a)  Develop  the  system  model  {Chapter  3) 

(1)  Mission  Objectives  and  Requirements 

(2)  Functional  Flow  Diagram 

(3)  Ev.nt  Sequencing  and  Operating  Times 

(4)  Syst  to  Operation  Modes 

(5)  Environmental  Profile 

(6)  Success/Failure  Criteria 

1 ' )  Logic  Representation 

(b)  Develop  a  formula  for  the  combination  of  individual 
failure  rates  (or  mean  times  between  failures)  of 
the  subsystems  or  components  to  derive  the 
reliability  of  the  system  (Chapter  4). 

(c)  Compile  Parts  lists  for  subsystems  or  components. 


•I 


(d)  Perform  Stress  Analysis. 


5-10 


(o) 

Ass ign 

failure  rates  to  parts. 

(f) 

Combine 

biiit 

part  failure  rates  to  determine  reli 
-  of  subsystems  or  components. 

(g) 

Compute 

system  reliability. 

2.4  COMPILATION  OF  PARTS  LISTS 

List  the  individual  parts  comprising  each  block  of  the  relia¬ 
bility  block  diagram.  Even  though  all  parts  of  block  are  listed 
only  those  parts  which  can  cause  the  failure  cf  a  block  are 
considered  in  the  reliability  prediction.  Parts  lists  wil’ 
serve  as  basic  worksheets  to  determine  stresses,  part  failure 
rates,  and  estimates.  When  entering  part  descriptions,  al  o 
record  ratings,  operating  voltages,  currents  and  power  dies. na¬ 
tion  . 

2.5  STRESS  ANALYSIS 

Record  on  the  worksheet  the  op  r~  ing  voltages,  currents  and 
other  char acter istics  needed  to  il  'ulate  stress  levels  of 
electronic  equipment  (for  mechanic?  equipment,  a  limited  amount 
of  data  is  available  correlating  the  rate  of  failure  with  stress 
levels ) . 

(a)  Determine  from  design  analysis  and/or  actual 
measurements  the  operating  voltages,  currents, 
power  dissipation,  etc.,  lor  each  part. 

(b)  Calculate  the  stress  levels  by  comparing  opera¬ 
ting  characteristics  or  cond: t ions  with  the 
rated  values. 

2.6  ASSIGNMENT  OF  PART  FAILURE  RATES  OR  PROBABILITIES 
OF  SURVIVAL 


This  step  in  the  reliability  prediction  consists  of  assigning 
failure  rates,  or  some  other  measure  of  reliability,  to  the 
individual  parts.  Most  part  failure  rate  dut  a  is  computed 
assuming  a  negative  exponential  distribution.  The  stress  levels, 
determined  in  the  stres  '  analysis,  ambient  temperatures,  and 
other  applicable  information  will  be  used  to  modify  or  adjust, 
these  failure  rates  for  use  in  a  particular  syste i  and,  or 
application .  If  the  stress  lew’s  or  the  environnenta  l 
characteristics  vary  during  a  mission,  separav  failure  rates 
'■vis  t  be  calculated  for  each  niasion  phase. 
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It  is  evident  that  a  key  factor  in  making  a  reliability  predic¬ 
tion  is  the  determination  and/or  availability  of  failure  rates . 
In  some  cases  the  failure  rate  of  an  equivalent  equipment  can  be 
obtained  directly  from  past  performance  data.  However,  the 
failure  rate  of  an  equipment  is  not  generally  available  in  the 
design  stage.  This  is  due  to  the  lack  of  operating  and  failure 
data  from  which  a  failure  rate  could  be  determined.  Therefore, 
the  determination  of  the  failure  rate  of  an  equipment  while  in 
the  design  stage  is  usually  based  on  the  details  of  the  design 
that  are  known,  i.e.,  types  of  parts,  ratings,  type,  duration 
and  magnitude  of  stresses  expected,  and  the  kind  of  operating 
and  failure  oata  from  which  the  expected  failure  rate  can  be 
determined . 

2.6.1  Sources  of  Part  Failure-Rate  Data : 

A.  Shipboard  Applications 

1 .  Handbook  for  the  Prediction  of  Shipboard  and  Shore 
Electronic  Reliability  by  R.  G.  Stokes  (NAVSHIPS 
93820,  Apr.  1961)  . 

2 .  A  Summary  of  Reliability  Prediction  and  Measure¬ 
ment  Guidelines  for  Shipboard  Electronic 
Equipment  (Vitro  Labs  Rpt.  #98,  Apr.  I'd57). 

3 .  Techniques  for  Reliability  Measurement  and 
Prediction  Based  on  Field  Failure  Data  ( V it.ro 
Labs  Rpt.  #80,  Oct.  1955). 

4 .  Study  of  Maintenance  Cost  Optimization  and 
Reliability  of  Shipboard  Machinery  (Unit ed 
Control  Corporation  Report,  June  1962), 

(AD2S3428 . ) 

B.  As  a  Function  of  Electrical  6.  Environmental  (External) 

Stress 

1 •  Reliability  Stress  Analysis  for  Electronic 

Equipment  (RCA  Rpt.  #TR-1100  or  NAVSHIPS  900-193 
Nov.  1956).  (RCA  Report  TR5q-  116-1  updates  this.) 

2 .  Philosophy  and  Guidelines  for  Reliability  Predic¬ 
tion  of  Ground  Electronic  Equipments  (RCA  Rpt. 

#R 4- 57  ,  Oct.  1957). 
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T.  C.  Reeves  (Military  Electronics ,  July  1957  )  . 

4.  PADC  Reliability  Notebook  (Depcrt  #RADC-TR- 
58-111). 

5 .  Reliability  Stress  Analysis  for  Electronic 
Equipments ,  M IL-HDBK-2 17 ,  31  Dec.  1961. 

6 .  Prediction  of  Field  Reliability  for  Airborne 
Electronic  Systems,  ARINC  Research  Corporation 
Publication  No.  203-1-344,  31  Dec.  1962. 

C.  Utilizing  Adjustment  (K)  Factors 

1 .  Investigation  of  Electronic  Equipment  Reliability 
(Aeronautical  Radio,  Inc.,  Air  Force  Reliability 
Assurance  Program,  Progress  Rpt.  #1,  Feb.  1956). 

2 .  Improved  Techniques  for  Design-State  Prediction 
by  H.  B.  Brown.  W.  C.  Fredrick,  and  H ,  J.  Kennedy 
(Air  Force  Reliability  Assurance  Program,  Progress 
Report  #2,  ARINC  Research  Corp. ,  Pub.  #110-1-136, 
Apr .  1959) . 

3  •  Reliability  ar.d  Maintainability  of  Military 
Electronic  Equipment  by  J.  H.  Hershey  (Bell 
Telephone  Labe,  3rd  Signal  Maintenance  Symposium, 
Apr.  1959). 

4.  Reliability  Analysis  for  Electronic  Equipment, 
Radio  Corporation  of  America,  TR-RQ-416-1 , 

Jan.  1959. 

5 .  "Component  Part  Failure  Rato  Analysis  for 
Prediction  of  Equipment  Mean  Life , "  R .  L . 

Vander  Hamm,  Collins  Radio  Co.,  CTR  195, 

March  1958. 

6 .  "Rel iabi 1 ity  Evaluation  Techniques  for 
Electronic  Equipment,"  Defense  Electronic 
Products  Division,  Radio  Co-  poration  of 
America,  Central  Engineering,  Camden,  N.  J., 

Vol.  14,  1962. 
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D.  Generic  Failure  Rates  &  Application  (K)  Factors 

1 .  Component  Part  Failure  Rates  Associated  with 
Installation  Environment  by  D.  E.  Earles 
(Mar t in- Denver  Report  &M60-47 ,  Dec.  1960). 

2 .  Reliability  Growth  Prediction  During  the  Initial 
Design  Analysis  by  D.  R.  Earles  (Proceedings  of 
the  7th  National  Symposium  on  Reliability  and 
Quality  Control,  January  1961). 

3 .  Reliability  Application  and  Analysis  Guide  by 
D.  R.  Earles  (Mar t in-Denver  Report  #M60-54, 
Failure  Rate  Handbook,  July  1961). 

4 .  Bureau  of  Naval  Weapons  Failure  Rate  Data  Hand¬ 
book,  (FAR  AD  A)  .  U.S.  Naval  Ordnance  Lab.,,  Corona, 
California.  (Available  only  to  qualified 
contractors  and  government  agencies.) 

5.  Failure  Rates,  D.  R.  trarles  and  M.  F.  Eddins, 

AVCO  Corporation,  April  1962.  (An  updated 
version  appears  in  Proceedings,  Ninth  National 
Symposium  on  Reliability  and  Quality  Control, 

Jan.  l°o 3  . ) 

6  .  Temco  Reliability  Manua 1  -  Vo'*  ♦  I ,  C  .  M .  S  chw  a  Lm  , 
Temuo  Electronics  and  Missiles  Company,  Dallas, 
Texas,  July  1961. 

E.  Mechanical  3nd  Electro-Mechanical  Devices 

1 .  Proposed  Procedures  fo r  Reliabi  1  ity  1  tress 
A~"-a  iyf.  i s  of  Mechanical  and  Electro-Mechanical 
Dev  ices  by  I.  Kirkpatrick  (RCA,  Ltd.,  Repom 
#P6,  Feb.  1958). 

2 .  Reliability  Analysis  Data _ for  Systems  and 

Component  Design  Engineers,  General  Electric 
Company,  Missile  and  Space  Vehicle  Department, 
Report  TRA-873-74,  distributed  by  U.  E.  Depart¬ 
ment  of  Commerce,  Office  of  Technical  Services, 
Washington  25,  D.  C.,  as  PB  181080. 

F.  Assigning  Reliability  Indices 
Prediction  of  Missile  Reliability  by  H.  R. 
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Powell  and  M.  J.  Kirby  (Sperry  Engineering 
Review,  Jul.-Aug.  1955). 

Active  Elements 

1 •  One  Reliability  Prediction  in  Satellite  Systems 
by  G.  T.  Bird  (ARINC  Research  Corp.  Pub.;  #4226- 
1-205,  May  1960) . 

2 .  A  Technique  Cor  Estimating  Ballpark  Reliability 
Figures  by  Tube  Counting  (RADC  Rpt.  #RADC-TN- 
58-81,  March  1958). 

Part  Variability 

1 .  Designing  Reliability  into  Electronic  Circuits 
by  A.  H.  Benner  and  B.  Meredith  (Proceedings 
of  the  National  Electronics  Conference,  vol. 

10,  1954). 

2 .  Circuit  Design  Concepts  for  High  Reliability 
by  F.  E.  Dreste  (Proceedings  of  the  6th 
National  Symposium  on  Reliability  and  Quality 
Control,  1960). 

3.  Statistics :  Key  to  Reliable  Military  Electronic 
Design  by  F.  E.  Dreste  (Military  Electronics, 
Vol.  VI,  No.  3,  March  1959). 

4.  The  Evaluation  and  Prediction  of  Circuit 
Performance  by  Statistical  Techniques  by 

S.  Marini  and  R.  T.  Williams  (Proceedings  of 
the  Joint  Military-Industry  Guided  Missive 
Reliability  Symposium,  Nov.  1957). 

5.  Designing  for  Reliability  by  S.  A.  Meltzer 
(IRE  Transactions  on  Reliability  and  Quality 
Control,  Sept.  1956). 

6.  Reliability  and  Components  Handbook  (Motorola 
Western  Military  Electronics  Center,  Jan.  1959). 

7 .  Electronic  Parts  Failure  Rates  Analysis  by  D.  J . 
Fisk,  Hughes  Aircraft  Company,  Aerospace  Group, 
Culver  City,  California,  Feb.  1963. 

8 .  Reliability  Data  Book  -  Engineering  Reliability, 
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Martin  Company,  Electronic  Systems  and 
Products  Division,  Baltimore,  Maryland, 

June  1962. 

These  sources  contain  part  failure  rates  based  on  part  character¬ 
istics  and  applied  stresses.  The  same  source  <~>f  failure  rates 
should  be  used  throughout  all  reliability  pre  -l.  tion  calcula¬ 
tions  (for  a  particular  system)  because  the  failure  rate  for  the 
same  part  may  be  different  in  each  of  the  sources.  This  is 
due  to  the  fact  that  the  failure  rates  in  each  source  are  not  based 
on  the  same  operating  conditions  and/or  failure  criteria.  These 
sources  categorize  parts  and  tubes  by  their  physical  character¬ 
istics  and  function.  Variations  in  failure  rates  are  presented 
as  a  function  of  stress  severity  expected  and  the  stress  level 
for  which  the  part  is  rated,  i.e.,  voltage,  power,  frequency, 
temperature,  actuation  rate,  speed  of  rotation,  etc. 

2.6.2  Failure  Rates :  Failure  rates  can  be  expressed  in  various 
ways : 

(a)  Failures  per  hour 

(b)  Percent  failures  per  thousand  hours 

(c)  Failures  per  thousand  hours 

(d)  Failures  per  million  hours 


(e)  Bits 


The  bit  is  usually  considered  to  be  the  minimum  failure  rate 
which  would  be  experienced  and  is  equal  to  1  x  10"8  failures  per 
hour.  j 


Table  1  is  provided  as  an  aid  in  converting  failure  rates  to  the 
desired  units.  To  use  the  table,  select  the  units  to  be  convert¬ 
ed  at  the  left  and  multiply  by  the  factor  at  the  intersection^, 
with  the  column  headed  by  the  desired  units,  e.g.,  to  convert  a\ 
failure  rate  of  1.4%  failures  per  thousand  hours  to  failures*  per 
hour,  multiply  1.4  by  10“  5  to  obtain  0.000014  failures  per  hour. 


ONVERS  ION  "N  ITS  F-  R  FAILURE  PATHS 
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2.6.3  Environmental  Stress  Correction;  The  availability  of 
dependable  failure  rate  data  is  essential  in  order  to  arrive  at 
a  meaningful  reliability  prediction.  Unfortunately,  the  data 
available  from  the  various  sources  are  based  on  dissimilar 
failure  criteria  and/or  different  use  environments.  It  is  for 
this  reason  that  the  tabulated  failure  rates  may  vary  consider¬ 
ably.  A  list  of  a  few  basic  failure  rates  from  thirteen  sources 
is  presented  in  Figure  5-17  .  An  inspection  of  the  table  shows 
that  it  is  not  uncommon  -o  nave  variations  of  three  orders  of 
magnitude  for  many  types  of  parts. 


Si 


For  example,  the  data  presented  in  MIL-HDBK-217  are  based  on 
three  classes  of  ground-based  equipments;  i.e.(  a  long-range 
search  radar,  a  communications  radio  set,  and  a  radar  identifica¬ 
tion  set.  Part  failure  rates  are  considered  to  apply  to  ground 
based  or  laboratory  bench  conditions.  In  comparison  the  data 
presented  in  NavShips  93820  was  based  on  average  severity  levels 
found  to  represent  several  dozens  of  equipment  types  used  in 
shipboard  applications.  AP.INC  Research  Report  203-1-344  was 
based  on  some  200  million  hours  of  operation  in  9  different  air¬ 
borne  «  v  c  t  e.m  c  . 


It  is  evident  that  care  must  be  followed  in  selecting  and  utili¬ 
zing  any  source  ct  failure  rate  data  for  a  specific  system  and/ 
or  application.  There  are  wide  variations  in  the  quality  of 
failure  and  data  analysis  as  well  as  the  effects  of  factors  such 
as  success/failure  criteria,  applied  stresses,  ard  operating 
environments.  Many  types  of  parts  do  not  have  derating  curves 
available.  The  FARADA  Handbook  offers  by  far  the  widest  selec¬ 
tion  of  data,  with  good  source  documentation.  Care  must  be  taken 


PART  FAILURE  RATE  COMPARISONS 

Failures  Per  Million  Hourr 


IANSISTOR  17.3  10.9  14.0  10.0  1.0  5.0-  0.1  1.03  9.0 

Germanium  r-  2.0  0.9  0.76 


FAILURE  RATE  DATA  SOURCES 
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when  using  FAR/DA  to  convert  failure  rates  tc  a  common  environ¬ 
ment  a1  base. 

In  many  reliability  prediction  procedures  the  basic  part  failure 
rates  must  be  modified  to  take  into  account  the  expected  environ¬ 
mental,  electrical,  mechanical,  and  thermal  stresses.  A  reasonable 
point  estimate  of  system  reliability  can  only  be  made  after 
extensive  stress  analysis.  The  predicted  reliability  may  not  be 
as  accurate  as  is  desired,  but  the  procedure  is  useful  in  focusing 
attention  on  potential  areas  of  unreliability. 

In  general,  correction  factors  will  take  a  form  similar  to  the 
following  equation: 

X  s  =  \;)  K~  K-,  . . . K j  ... Kn  )  , 

wher r-:  \  a  is  the  adjusted  failure  rate,  >,  is  the  basic,  or  gen¬ 
eric  failure  rate,  and  K,  represents  the  correction  factors 
needed  to  modify  the  basic  failure  rate  due  to  differences  in 
applied  stresses,  ratio  of  likely  tolerance  failures  to  random 
catastrophic  failures,  external  environments,  maintenance  pra¬ 
ctices,  complexity,  observed  cycling  effects,  etc. 

Reliability  prediction  techniques  vary  in  the  degree  of  utiliza¬ 
tion  or  consideration  of  correction  factors. 

a.  AVCO  method  (Report  listed  in  paragraph  2.6.1  D3) :  In 
order  to  predict  the  failure  rate  of  a  system,  the  parts  generic 
failure  rates,  which  have  been  normalized  to  laboratory  computer 
conditions,  are  multiplied  by  application  or  derating  factors 
and  then  by  factors  which  represent  the;  installation  environment. 

b.  MIL  HDBK  217  method  (Reliability  Stress  Analysis  for 
Electronic  equipments):  To  obtain  a  failure  rate  prediction 
for  a  system  by  this  method,  the  parts  basic  failure  rates  are 
modified  by  expected  electrical  and  thermal  factors  and  then 
further  modified  by  a  factor  related  to  the  environment  in 
which  the  system  is  expected  to  operate. 

The  following  table  compares  the  environmental  factors  used  in 
the  AVCO  method  with  Lhose  used  in  the  MIL  HDBK  217  method. 
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TABLE  2 


Installation 

i  Environment 

Environmental 

Correction  Factors 

A^CO 

MIL  HDBK  217 

Shipboard 

15.0 

1.0 

Ground 

8.0 

1.0 

Aircraft 

50.0 

6.5 

Missiles 

900.0 

80.0 

Satellites 

Launch  Phase 

900.0 

80.0 

Boost  Phase 

800.0 

80.0 

Orbit  Phase 

1.0 

1.0 

2.6.4  Special  Cases;  Data  applicable  to  parts  whose  failure 
rates  change  with  time,  to  one-shot  devices  and/or  to  parts 
whose  probabilities  of  survival  do  not  depend  on  time,  should 
be  recorded  in  the  form  of  a  probability.  If  the  probability  of 
survival  is  time-dependent,  the  corresponding  value  must  be 
recorded  for  each  of  the  time  periods  under  investigation. 

2.7  COMBINING  PART  FAILURE  RATE  TO  OBTAIN  SYSTEM  OR  COMPONENT 
RELIABILITY  ‘ 


In  the  Radar  example  of  Chapter  3,  an  example  of  redundant  use 
of  controls,  transmitters,  receivers  and  indicators  was  shown. 

The  logic  diagram,  at  the  bottom  of  the  chart  (Figure  3-19  )  shows 
the  alternate  paths  that  would  constitute  success,  similar  to 
the  example  in  Chapter  4,  (Figure  4-37).  An  alternate  method  of 
a  mapping  technique  for  solving  problems  of  combined  series- 
parallel  probabilities  is  given  in  Reference  10.  The  logic  dia¬ 
gram  shown  describes  the  rule  of  combination  of  probabilities 
for  the  combined  system.  For  each  component  (controls,  synchro¬ 
nizer,  transmitter)  the  subassemblies  must  be  identified  and  a 
block  diagram  constructed  to  show  the  interrelationships  so  that 
any  redundant  sections  can  be  identified.  It  is  not  usually 
warranted  to  attempt  to  evaluate  redundancies  between  parts, 
(transistors,  capacitors  or  relays),  since  in  the  usual  design, 
such  redundancies  will  have  a  relatively  insignificant  effect  on 
the  system  reliability.  In  some  special  cases,  where  such  re¬ 
dundancy  is  employed  as  a  reliability  improvement  technique  to 
solve  a  specific  problem,  it  can  and  should  be  computed. 

The  failure  rate  of  a  block  which  contains  only  parts  in  series 
having  constant  failure  rates  is  the  sum  of  the  parts  failure 
rates.  To  obtain  the  failure  rate  of  a  block  containing  redun¬ 
dant  groups  of  parts,  or  parts  which  do  not  have  constant  failure 
rates,  substitute  the  part  failure  rates  or  probabilities  in  the 
block  reliability  formula  developed  from  the  system  model. 
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3 •  COMPONENT  RELIABILITY  PREDICTION 

Figures  5-22  and  5-23  demonstrate  the  computation  of  reliability 
of  a  component.  All  parts  are  considered  in  series,  that  is,  a. 
failure  of  any  part  will  cause  a  failure  of  the  entire  component. 
The  failure  rates  of  the  individual  parts,  corrected  for  applica¬ 
tion  stress  and  environmental  factors  are  added  together  to 
obtain  a  failure  rate  which  is  converted  to  the  MTBF  shown. 

NAVSHIPS  93820  provides  a  more  comprehensive  example  of  predic¬ 
tion  of  reliability.  Xt  establishes  four  levels  of  reliability 
prediction  for  electronic  equipment  based  on  the  degree  of  know¬ 
ledge  of  the  system.  Method  D,  the  most  comprehensive,  applies 
derating  (or  load  factors)  to  the  parts  based  on  application 
data. 

As  previously  mentioned,  the  prediction  of  reliability  of  mech¬ 
anical  systems  in  lagging  far  behind  the  electronic  systems. 

Some  data  on  expected  failure  rates  for  mature  (well  developed) 
mechanical  components  is  available  in  the  literature.  This  must 
be  used  with  caution,  but  can  be  used  with  engineering  judgment. 

If  a  proposed  hydraulic  system,  for  example,  is  about  the  size 
and  sees  about  the  same  load  factors  as  the  hydraulic  components 
in  an  airplane,  data  from  airplane  experience  (FARADA  for  example) 
can  be  used.  Where  the  sizes  are  much  greater  and  the  loads  less, 
the  values  given  may  be  extremely  pessimistic.  In  this  case, 
personal  experience  and  consultation  with  suppliers  of  hydraulic 
components  typical  of  the  proposed  system  will  provide  a  better 
guide. 

A  prediction  is  an  estimate  of  achievable  reliability.  Engineer¬ 
ing  judgment  may  in  many  instances  be  superior  to  available  data. 
If  the  purpose  of  achieving  high  reliability  i3  to  be  served,  the 
engineer  must  seek  out  the  facts  and  apply  sound  judgment  to 
their  interpretation.  See  Chapter  12  for  a  more  comprehensive 
approach  to  the  prediction  of  reliability  of  mechanical  systems. 

4 .  EXAMPLE  OF  SYSTEM  RELIABILITY  PREDICTION 

4.1  TYPICAL  SYSTEM 

The  first  step  in  calculating  the  system  reliability  is  to  obtain 
a  reliability  estimate  of  the  individual  subsystems.  The  follow¬ 
ing  reliabilities  will  be  assumed  for  illustrative  purposes 
(Figure  5-24) : 
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Step  1:  Reduce  the  parallel  combination  of  items  9,  10,  and  11 
to  single  Item  A,  ir.  series  with  the  remaining  circuit. 
(Figure  5-261 

RA  =  1  -  (1  -  R9>  (1  -  P.iQ)  (1  -  Ru> 

=  1  -  (1  -  0.9983)  (1  -  0.9980)  (1  -  0.9967) 

=  0.9999  * 

Step  2:  Reduce  the  series  parallel  combination  of  Items  6,  7, 
and  8  to  a  single  Item  u. 

R  -l-(l-R)  •  (1-RR) 

Li  Vo 

P  1  -  (1  -  0.9980  rl  -  (0.9980)  (0.9989)] 

-  0.9999  + 

St°p  0:  Reduce  the  series  combination  of  Items  2  and  3  to  a 
single  Item  L. 

RL  =  R2  X  R3 

=  0.9950  x  0.9967 

=  0.9917 

Reduce  the  series  combination  cl  items  4,  5,  and  u  *-o 
a  single  Item  S. 

R_  =  R.  x  R,.  x  R 
S  4  5  u 

-  0.9975  x  0.4*17  x  0.9999  + 
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=  0.9961  + 

Step  4:  Reduce  the  parallel  combination  of  Items  L  and  S  to  a 
single  Item  B. 

R  =  1  -  (1  -  Rr)  *  (1  -  Rc) 

3  1_j  o 

=  1  -  (1  -  0.9917)  (1  -  0.9961) 


=  0.9999  + 

Step  5:  By  reducing  the  series  combination  of  Item  1,  B,  and  A 
we  get  the  overall  system  reliability,  R  .  . 


Rtotal  ”  R1 


R. 


R 


B  A 

=  0.9980  x  0.9999+  x  0.°999+ 


*  0.9978  + 


The  reliability  of  a  system  composed  of  two  parallel  redundant 
branches,  each  containing  two  series  subassembxies  would  be  cal¬ 
culated  from  the  following  equation:  (Figure  5-28). 
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(1  -  R 


Ac 


V 


A  system  composed  of  two  series  sets  of  parallel  redundant  sub- 
assemblies  would  have  a  reliability  given  by  the  following 
equat ion : 


1  -  (1  - 


V 


(1  - 


RA2> 


j[l 


-  (1  - 


rbi> 


(1  - 


RB2J1 


Assuming  only  a  non-transmitting  mode  of  failure,  equal  relia¬ 
bilities  of  the  corresponding  components  in  Systems  I  and  II, 
and  that  there  is  ho  physical  interaction  that  would  change  the 
system  relj  ability,  and  giver. 


then 


0.950 


0 . 900 


r  =  1  ..  [1  _  (1  -  0.950)  (0.900)  ][1  -  (0.950)  (0.900)] 

s 

-  0.979  for  System  I, 


and 


R  -  [1  -  (1  -  0.950) (1  -  0.950) ][1  -  (1  -  0 . 900) ( 1-0 . 900) ] 
=  0.988  for  System  II. 


REDUNDANCY  CONSIDERATIONS 
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'It  is  possible  to  demonstrate  mathematically  that  under  the 
given  assumptions  the  reliability  of  SYSTEM  II  is  better  than 
the  reliability  of  SYSTEM  I.  The  numerical  results  above  confirm 

this  point. 

4.2  EXCEPT lOHS  TC  SERIES  PARALLEL  SOLCTIONS 

All  reliability  problems  cannot  be  reduced  to  clear-cut  cases  of 
series,  parallel  or  stand-by  models.  Consider  the  case  illus¬ 
trated  in  Figure  5-30. 

A  and  A '  are  in  series  and  so  are  B  and  B?.  Paths  A-A '  and  B-B7 
are  in  parallel,  so  that  an  output  is  present  if  at  least  one  path 
is  functioning  properly.  However,  to  improve  reliability,  unit 
C  is  added.  Its  function  is  to  supply  A '  or  B;,  if  necessary, 
when  an  ?jppropriate  signal  is  received.  C  is  not  in  parallel 
with  A  or  Be  and  hence  the  circuit  will  not  resolve  to  a  simple 
parallel-series  combination.  ■ 

To  solve  the  problem  on  hand,  use  can  be  made  of  Bayes  probabil¬ 
ity  lemma,  which  in  terms  of  reliability,  states: 

Q  =  Q  {.if  C  is  good)  R  +  Q  (if  C  is  bad)  Q 
&  s  c.  s  c 

where  Q  denotes  the  probability  of  system  failure 

S 

R  denotes  the  reliability  of  block  C 

Q  denotes  the  probability  of  failure  of  block  C 

In  other  words  this  the<kem  states,  that  the  probability  of 
failure  of  the  complete  system  (no  output)  is  the  probability  of 
the  system  failing  if  C  is  good,  times  the  reliability  of  block 
C  plus  the  probability  of  system  failure  if  C  is  bad  times  the 
probability  of  C  failing. 

Mow,,  if  c  is  good,  the  system  will  fail  only  if  both  A '  and  B* 
fail.  A /  and  B '  being  in  parallel,  the  probability  of  system 
failure  (the  unreliability  of  the  system)  is  then: 

Q  (if  C  is  good)  =  (1-R  /)  (1-R  /) 

S  A  B 

If  C  is  bad,  the  system  reduces  to  a  common  series-parallel 
system  and  the  probability  of  system  failure  is: 

Q  (if  C  is  bad)  -  (^-R.R.-)  (l-RQhn • ) 

S  A  A  B  B 


SERIES  -  P 
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where  ( 1 — R  R  / 1  is  the  unreliability  of  A-A  series  path,  and 
A  A  1 

( 1-R  R  / )  is  the  unreliability  of  B-b'  series  path.  The  unrelia- 
B  B 

bility  of  the  whole  systeir  can  now  be  written: 

Q  =  ( 1-R  ' )  (1-R  /)  R_  +  (1-R  /R  /)  (l-R/R  ,)  (1-R_) 

s  A  BC  AA  BB  u 

Hence  the  reliability  of  the  system  is: 

Rs  =  1-QS  =  MC(U|,I  <1-Rb-)  -  (1-RC)  n-RARA'>  '‘-Vb'1 

In  order  ^.o  illustrate  further  the  application  of  Bayes  Lemma, 
consider  the  following  example; 

Example:  A  30KV,  GO  cps  transmission  line  comes  to  a  paper  mill 

area  and  is  there  stepped  down  to  a  440V  by  two  main  distribu¬ 
tion  transformers,  blocks  A  and  B  (See  Figure  5-32).  These 
blocks  also  have  circuit  breakers  associated  wi*h  trans¬ 

formers  and  required  to  protect  the  transformer  in  case  of  short- 
circuit  or  overload. 

We  assume  in  this  case  that  the  reliability  of  the  transmission 
line  is  100%.  Hence  we  can  consider  A  and  B  as  two  independent 
sources  of  power.  a'  and  B* include  the  distribution  equipment 
(circuit  breakers,  cabling,  bus,  etc.)  for  power  supplies  A  and 
B  respectively.  The  outputs  of  a'  and  B*  are  connected  together 
and  thus  feed  the  load  in  parallel. 

As  in  most  process  industries,  a  power  failure  is  rather  critical 
and  will  cause  extensive  losses,  because  re-starting  of  the  plant 
after  recovery  of  power  cannot  be  immediately  effected.  There¬ 
fore  reliability  must  increased  considerably  by  adding  a  third 
power  source,  C,  which  ..unctions  as  a  standby  for  both  primary 
sources.  C  is  a  diesel-engine  or  steam-turbine-driven  three 
phase  alternator,  with  its  two  circuit  breakers,  CBl,  CB2.  Now, 
without  block  C,  if,  for  instance,  A  fails,  B  will  feed  the  load, 
but  the  parallel  redundancy  is  lost.  With  C  in  the  circuit  a 
parallel  redundancy  is  maintained  if  either  A  or  B  malfunctions. 

Assuming  the  following  reliability  values  for  the  blocks  A,  A, 

B,  B  and  C,  let  us  compute  the  system  reliability  for  the  cases 
without  and  with  block  C. 

Let  R  =  R„  =  0.8 

A  B 

R  /  =  R  /  =  0.9 
A  B 


0.8 


POWER  DISTRIBUTION  SYSTEM 
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a)  Without  block  C 

Q  --  ( 1  -R  R  ,)  (1-R  P.  /)  -  (1-0.0  x  0.9)  (1-0.8  x  0.9) 

s  A  A  B  B 

-  0.078 

R  =  1-Q  =  1-0.078  =  0.922 

s  s 

b)  Block  C  included 

R  =  1-R  (1-R  /) (1-R,)  -  (1-R  1*  (1-R  R  ,)  (l-RJ’  ') 
s  CA  B  C  AA  BB 

-  1-0. 8(1-0. 9) (l-0.9)-(l-0.8)[l-(0.8) (0.9) ][  1- ( 0 .8 ) (0.9) ] 


=  0.976 


As  ^an  be  seen  the  reliability  of  the  system  has  improved  about 
5.4%,  after  block  C  was  inserted.  By  comparing  the  decrease  in 
unreliability,  the  improvement  appears  even  more  dramatic.  The 
'inr-eliabilities  in  the  two  cases  are: 


a) 

Q 

=  0.078 

l! 

00 

* 

s 

b) 

Q 

=  0.024 

=  2.4% 

In  other  words  the  system  unreliability  has  dropped  from  7.8%  to 
2.4%,  a  factor  of  3.25. 

4.3  EXAMPLE  OF  RELIABILITY  OPERATIONAL  MODEL 

Figure  5-34  illustrates  the  power  generation  section  of  a  ground 
support  electrical  system  for  a  missile  site.  In  order  to  evalu¬ 
ate  the  system  reliability,  failure  rate  data  for  all  equipment 
in  the  system  must  be  known  or  estimated. 

The  system  will  be  broken  down  to  subsystem  or  components  for  the 
purpose  of  establishing  RFB's  (Reliability  Functional  Blocks). 

The  RFB's  should  be  composed  of  components  or  subsystems,  which 
are  replaceable  i..  the  field.  In  the  example,  the  RFB’s  shown 
in  Figure  5-35  could  be  used. 

4.3.1  RFB  Descriptions : 

1.1  Diesel  Engine:  This  function  consists  of  the  diesel  engine, 
engine  instrumentation  board  and  other  apparatus  needed  for 
control  and  monitoring  the  operation  of  the  diesel  engine. 


POWER  GENERATION  SECTION 


FUNCTIONAL  MODEL 
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1.2  480  VAC,  60  CPS  Generator:  This  function  is  defined  as  the 
aenerator  itself,  its  controlling  equipment  and  the  appara¬ 
tus  necessary  to  transfer  the  generated  power. 

1.3  Circuit  Breaker:  This  block  contains  that  portion  of  the 
switchgear  which  carries  the  electric  power  from  RFB  1.2  to 
the  distribution  bus. 

2.1  Incoming  Commercial  Power:  This  function  is  defined  to  be 
that  portion  of  the  commercial  power  system  to,  and  inclu¬ 
ding,  the  switch  on  the  power  pole. 

2.2  High  Voltage  Cabling:  That  section  of  the  wire  from  pole 
switch  to  the  power  transformer. 


2.3  Power  Transformer:  The  step-down  transformer. 

2.4  Circuit  Breaker:  That  portion  of  the  switchgear  which 
carries  the  commercial  p.  ■°r  to  the  main  bus. 

2.5  Diesel  Engine  Starting  Contactor:  This  block  consists  of 
the  diesel  engine  starting  contactor  and  the  w'iring  connec¬ 
ting  it  from  the  low  side  of  step  down  transformer. 

4.3.2  Logic  Diagram:  Having  the  -eliab’lity  functional  block 
diagram  and  the  description  of  the  RFB’s,  we  are  able  to  con¬ 
struct  the  mathematical  model  for  the  system  under  study.  The 
model  consists  of  a  logic  diagram,  ana  equations  giving  the  re¬ 
liability  in  terms  of  failure  rate  <  and  operating  time  t  of 
each  functional  block  (RFB). 


4.3.3  Reliability  Equation:  This  problem  is  a  case  of  stand¬ 
by  redundancy  with  repair.  Therefore  the  reliability  for  time 
t  may  be  stated: 


R  =1  ~  ) 
s 


{(*■2.1  4 


t-T  (1-1)  ]}' 


-U2.1  +  W*  1.-1  >  i  t-T  ( l-l)  ] 


\  ,  \  „  >  ^  4 

-(-$•-  +  1 

(1  -  e  “  ) 

-  _  +  iiUL  ,  *  f  l  _ 

'  2  2  7'  1 


where  T  is  the  mean  time  to  repair  for  blocks  2. 1-2.3. 
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4.3.4  Failure  Rates:  It  is  assumed  that  the  following  data 
for  failure  rates  have  been  collected  from  tests  and  previous 
case  histories  of  component  failure. 


.  =  4. 088 3 C  failures/ 1000  hours 
■ , -  -  0.4088  3  fai lures/ 1000  hours 
=  0.1300  failures/1000  cycles 
v?  j  -  0.68433  failures/1000  hours 

i.pj  -  0.11415  failures/ 1000  hours 

>w,:,  =  0.22831  failures/1000  hours 

\?4  =  0.10000  failures/1000  cycles 

=  0.66667  failures/1000  cycles 
T  =  2.000  hours 

4.3.5  Operating  Times:  Definition  of  the  operating  conditions 

for  each  RFB  is  generally  required. 

1.1  Diesel  Engine  I:  This  unit  is  operating  during  the  inter¬ 
val  when  it  is  started  until  it  is  stopped. 

1.2  400  VAC,  60  CPS  Generator  I:  This  unit  is  considered  oper¬ 
ating  whenever  RF8  l.l  is  operating. 


1.3  Circuit  Breaker  I:  This  unit  operates  whenever  there  is  a 
failure  of  the  commercial  power  or  when  it  is  manually 


operated.  A  complete  cycle 
the  breaker  to  OFF  and  back 

2.1  Incoming  Commercial  Power: 
operating  whenever  there  is 
for  use  at  the  site. 


is  defined  as  the  movement  of 
to  ON  position. 

This  function  is  considered 
commercial  power  available 


2.2  High  Voltage  Cabling:  This  function  is  considered  operating 
whenever  RFB  2.1  is  operating. 


2 . ?  Power  Transformer:  This  function  is  considered  operating 
whenever  RFB  2.2  is  operating. 

2.4  Circuit  Breaker:  This  unit  operates  simultaneously  with 
RFB  1.3. 

2.5  Diesel  Engine  Starting  Contactor:  This  unit  is  considered 
operating  whenever  RFB  1.3  operates.  It  is  also  a  cyclic 
function. 


hs 


4.3.6  Time  Bar  Graph:  From  the  definitions  above,  we  can  now 
construct  the  time  bar  graph  (Figure  5-29).  Note  that  blocks 
1.3,  2.4  and  2.5  operate  only  at  fault  in  the  primary  (commercial 
power)  system  and  at  its  recovery.  In  this  case,  there  is  no 
function  which  is  turned  "off”  or  "on".  However,  this  is  not 
the  usual  occurrence  and  in  cases  where  switching  takes  place,  it 
is  less  confusing  and  easier  to  :  ake  the  model  if  a  time  bar 
graph  is  used, 

4.3.7  System  Reliability:  Substituting  the  failure  rate  values 
and  operating  times  in  the  pertinent  equations,  the  system  re¬ 
liability  for  an  operating  period  of  30  days  is: 


R  =1-  ) 
s  u 

i=l 


,$oFjl. 02739[0. 720-0. 002  (i~l)  l}1  }  Q27 


39[0. 720-0. 002(i-l)  ] 


1  -  e 


-0.76667i 


-0.76667i  ..  -4.49713  x  0.00C2i, 

+  e  ( 1-e  )| 

-  0.9928 

For  the  sake  of  illustration,  let  us  compare  the  reliability  of 
system  2.0  above,  that  is  without  the  standby  redundancy,  to  that 
just  computed.  The  reliability  without  standby  redundancy  is 

,  -(\a,  +  X,.a  +  \s.3)t  -(0.68494  +  0.11415  +  0.22831)0.72 


R(s)  =  e 


-0.73972 


0.4772 


Thus  the  reliability  has  increased  from  47.72%  to  99.28%  by  using 
standby  redundancy  with  repair. 


SUMMARY 


The  purpose  of  reliability  predictions  is  to  arrive  at  a  numerical 
evaluation  (quantitative)  of  the  reliability  potential  of  a 
system,  equipment,  etc.,  and/or  to  determine  whether  or  not  a 
specific  system,  equipment,  etc.,  will  meet  its  predetermined  or 
required  reliability  goal.  It  is  necessary  to  perform  these 
reliability  predictions  during  the  design  stage.  This  enables 
the  design  to  be  evaluated  in  terms  of  reliability  and  allows 
design  changes  which  may  be  needed  to  improve  reliability  to  be 
made  at  this  early  stage  where  it  is  most  economical  as  well  as 
convenient. 


TIME  BAR  GRAPH 
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It  should  be  remembered  that  reliability  predictions  can  be  made 
at  various  complexity  levels.  The  selection  of  method  to  be 
used  is  determined  by  such  factors  as  required  accuracy,  time 
available,  cost,  etc.  However,  no  matter  which  method  is  used  in 
calculating  a  reliability  prediction,  it  must  be  based  on  design 
details  and  on  reliability  or  failure  rate  data  of  equipments 
or  parts  of  similar  design  and  under  similar  operating  conditions 
of  stress,  time,  and  environment. 


5 .  RELIABILITY  GROWTH  APPROACH 

The  reliability  prediction  as  made  is  called  inherent..  This 
refers  to  some  future  time  when  the  design  is  matured  --  has  all 
the  weaknesses  and  defects  due  to  manufacturing  cleared  out. 

We  are  bound  to  be  concerned  with  the  reliability  achieved  during 
the  development.  We  are  particularly  concerned  that  we  obtain 
visibility  of  reliability  growth  toward  the  deliverable  require¬ 
ment.  Recognizing  that  the  growth  process  is  the  process  of 
isolating  and  eliminating  weaknesses,  there  is  a  real  need  to 
evaluate  progress  toward  the  goal.  In  dealing  with  contractors 
we  can  expect  some  to  develop  growth  predictions  as  a  basis  for 
further  development  or  continuation  of  a  project. 

The  promise  of  improvement  can  be  validated.  In  evaluating  such 
promise  we  must  look  at  the  physical  basis  on  which  it  is 
founded.  The  basic  philosophy  and  foundation  of  growth  models 
is  provided  to  give  a  basic  understanding  of  the  concepts. 

5.1  RELIABILITY  GROWTH  MODELS 

One  approach  to  construction  of  a  reliability  growth  model  is  to 
postulate  the  form  of  the  function  relating  reliability  and  time, 
say  R  -  f(t).  The  argument  t  could  represent  any  index  of  relia¬ 
bility  growth  such  as  the  number  of  reliability  tests  conducted 
on  the  device,  the  time  since  the  development,  of  the  device 
began,  the  amount  of  money  invested  in  development,  eve. 

Having  assumed  such  a  relationship,  one  proceeds  to  estimate  the 
parameters  of  the  function  by  some  curve  fitting  technique, 
analogous  to  the  procedures  utilized  in  linear  regression.  When 
this  approach  is  taken,  a  function  having  the  following  properties 
is  usually  selected. 

f(t)  is  nondecreasing  in  t  (reliability  growth) 

f(t)  approaches  R  as  t  approach  ®  where  R  is  the  maximum 

nr.  ix, 

attainable  reliability.  A  function  meeting  this  criterion  is, 
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_  Kt. 

for  example,  R { t )  -  -  C 1  e  .  This  particular  form  of 

the  equation  states  that  the  reliability  at  time  t,  R(t)  is 
limited  by  the  inherent  (predicted)  reliability  and  will  approach 
that  value  r s  tests  are  conducted  at  a  rate  proportional  to  the 
gap  between  the  actual  and  predicted  reliabilities. 

In  development  prog  -ams  it  is  generally  supposed  that,  given  a 
basic  design,  reliability  can  be  improved  through  a  "test  and 
fix"  procedure.  That  is,  as  causes  of  failure  are  detected  by 
tests,  action  can  be  taken  tc  correct  the  cause  of  the  failure*  - 
action  which  is  not  always  effective.  The  causes  of  the  greater 
numbers  of  failures  w ill  probably  be  detected  early  in  the  pro¬ 
gram,  and  consequently,  have  the  greater  chance  of  being  correc¬ 
ted  early.  This  tends  to  justify  the  general  form  of  the  relia¬ 
bility  growth  curve  in  which  reliability  increases  rapidly  in 
the  early  stages  with  the  rate  of  increase  diminishing  with 
time.  The  notion  of  inherent  design  reliability,  i .e.,  the 
limit  which  the  growth  function  approaches  in  time,  can  be  jua- 
tified  in  a  similar  manner.  Designs  which  inherently  contain 
many  causes  of  failure,  most  of  which  will  cause  only  a  small 
number  of  failures,  would  have  a  low  design  reliability.  After 
the  principal  causes  of  failure  are  eliminated  early  in  the  pro¬ 
gram,  each  "test  and  fix"  cycle  will  only  improve  reliability  a 
very  small  amount.  Eventually,  the  reliability  curve  will  tend 
to  level  out.  This  level  approaches  the  inherent  design  relia¬ 
bility. 


Another  approach  to  a  growth  model  is  given  in  reference  (1).  A 
fixed,  but  unspecified,  number  of  failure  modes  are  allowed  to 
exist.  Whenever  a  failure  mode  is  discovered  by  test,  an  attenpt 
is  made  to  correct  the  cause.  The  probability  that  the  correc¬ 
tive  action  is  successful  for  the  ith  failure  made  is  a  known 
quantity  a( .  That  is,  the  probability  of  correcting  the  ith 
failure  mode,  given  that  such  a  failure  has  occurred  on  test,  la 
a,.  In  this  model,  N  tests  are  conducted  prior  to  taking  any 
corrective  action.  The  reliability  of  the  system  after  N  testa 
have  been  made  and  corrective  action  has  been  taken  for  all 
detected  failure  modes  is 


where 


N 


K 

rq  +  )  y i^i 
i“l 


^0  if  N(  -  0 
a,  if  N,  >  0 


i 
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N, 

K 

qt 

R0 


K 

total  number  of  failures  observed  in  N  tests  =  ^  Nj 

i-1 

number  of  failures  of  the  ith  mode  observed  during  test 
number  of  failure  modes 

probability  of  a  failure  of  the  ith  mode 
initial  reliability  of  system. 


It  is  assumed  that  a  given  test  can  result  in  success  with 
unknown  probability  R0 ,  or  in  failure  oy  only  one  of  the  K  failure 
modes  and 

K 

R0  +  T  q,  =  1. 

i — i 

i  =  l 

The  parameters  of  this  model  are  R0  and  the  K  q5  ' s .  The  a,  are 
assumed  known.  The  random  variables  resulting  from  tests  are  N0 , 
N-,  ,  N?  ,  . . . ,  Ny .  It  is  assumed  in  the  analysis  that  the  tests 

are  independent.  It  is  easily  seen  that  this  model  is,  in  a 
sense,  a  generalization  of  the  Lloyd  and  Lipow  model  (2)  in  that 
reliability  growth  is  obtained  bv  taking  credit  for  having  cor¬ 
rected  some  of  the  original  causes  of  failure. 


From  a  practical  point  of  view,  this  model  has  some  real  value. 

It  appears  to  be  a  reasonable  representation  of  some  real  world 
situations,  and  its  use  requires  input  data  which  in  many  cases 
will  oe  aval1 able.  It  is  not  too  hard  to  envision  situations 
where  an  engineer  can,  based  on  his  previous  experience,  estimate 
fairly  accurately  the  probability  (as)  that  a  corrective  action 
will  be  effective.  It  should  be  noted  that  this  estimate  is 
required  only  when  a  corrective  action  is  actually  taken.  It  is 
significant  that  both  of  dnese  decisions  (selection  of  the  a, 
and  the  likelihood  that  a  corrective  action  will  introduce  other 
modes  of  failure)  can  be  framed  in  physical  terms  as  engineering 
questions . 


5.2  APPLICATION  OF  GROWTH  APPROACH 

The  use  of  reliability  growth  approaches  promises  the  gradual 
elimination  of  quality  defects.  They  are  effective  insofar  as 
the  trend  toward  higher  reliability  improves.  They  should  be 
used  with  caution  unless  solid  engineering  or  test  data  eoniirm 
that  the  growth  is  real. 
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I  6.  PURPOSES  FOR  RELIABILITY  PREDICTION 

A  primary  means  of  establishing  the  reliability  feasibility  of  a 
design  concept  is  the  comparison  of  pre-design  predictions  with 
requirements.  The  consideration  of  the  direction,  magnitude,  and 
causes  of  discrepancies  between  predictions  and  requirements 
plays  an  important  role  in  determining  proper  courses  of  action. 

A  basic  problem  in  reliability  prediction  can  be  generally  des¬ 
cribed  as  follows:  Given  a  complex  device  or  system,  such  as  a 
communication  set,  fire  control,  sonar  or  computer,  at  some  stag* 
of  design  and  development,  we  are  interested  in  whether  or  not  it 
will  function  in  a  given  environment,  in  a  prescribed  manner,  and 
for  a  given  period  of  time.  Whether  the  system  operates  success¬ 
fully  or  not  depends  on  a  very  large  number  of  factors.  When 
predicting  the  reliability  of  a  given  complex  equipment,  if 
possible,  one  should  evaluate  the  interactions  present  since  these 
may  have  a  preponderant  effect  on  the  over-all  system  reliability. 

Reliability  predictions  aid  in  the  identification  and  solution  of 
problems  that  are  broad  in  scope  and  general  in  nature.  This  is 
accomplished  by  the  tabulation  and  grouping  of  predicted  relia¬ 
bilities,  or  unreliabilities,  for  specific  Dart  tvoes .  part 
classes,  and  equipment  types,  and  for  operation  in  various  modes 
or  during  different  phases  of  a  mission.  The  knowledge  of  the 
relative  contribution  of  various  items  of  equipment  and  modes  of 
operation  to  the  systems  unreliability  constitutes  a  sound  basis 
for  determining  the  need  for,  and  the  expected  benefits  of,  part 
improvement  programs,  circuit  and  equipment  redesign  efforts, 
inclusion  of  redundancy,  reallocation  of  requirements,  and  othmr 
similar  courses  ol  action. 

Another  valuable  use  of  reliability  predictions  is  to  focus 
attention  on  items  for  which  adequate  design  data  are  not  avail¬ 
able.  It  will  frequently  be  found  that  necessary  failure  inform¬ 
ation  is  rot  available.  This  is  especially  true  for  new  parts 
and  parts  peculiar  to  a  specific  application.  The  process  of 
reliability  prediction  uncovers  these  data  deficiencies  and 
permits  early  planning  for  corrective  action,  i.e.,  revision  of 
specification,  selection  of  a  different  part,  starting  a  data 
collection  program,  or  performing  special  tests. 

An  obvious  purpose  of  reliability  predictions  is  to  serve  as  s 
means  to  measure  progress  in  achieving  a  reliability  goal,  i.e., 
comparison  of  predictions  with  previous  predictions  to  s*s  whether 
a  program  is  progressing  satisfactorily  or  not.  If  the  progrm 
is  progressing  satisfactorily,  it  ma^  be  decided  that  tne  activ¬ 
ities  should  continue  as  planned.  However,  if  progress  is  not 


I 


4 


satisfactory,  the  predictions  may  be  used  to  determine  what 
action  should  be  taken  as  well  as  where  the  re-emphasis  should 
be. 
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Chapter  6 
APPORTIONMENT 

System  design  engineers  must  translate  overall  system  character¬ 
istics,  including  reliability,  into  detailed  specifications  for 
the  numerous  units  that  make  up  the  system.  The  process  of 
assigning  reliability  requirements  to  individual  units  to  attain 
the  desired  system  reliability  is  known  as  "reliability  appor¬ 
tionment"  or  sometimes  termed  reliability  allocation.  More  is 
involved,  however,  than  a  simple  mathematical  equality.  The 
reliability  of  an  individual  unit  varies  with  the  type  of  func¬ 
tion  to  be  performed,  the  complexity  of  the  unit,  and  the  method 
of  accomplishing  the  function,  to  name  a  few  of  the  more  impor¬ 
tant  factors.  The  role  a  unit  plays  in  a  particular  system  also 
enters  into  consideration. 

Apportionment  of  system  reliabi lity . is  the  inverse  process  to  a 
reliability  prediction.  In  a  prediction  we  estimated  failure 
rates  of  parts  and  subsystems  (or  numbers  of  failures  per  unit 
of  time)  and  computed  a  system  failure  rate  (total  number  of 
failures  estimated  per  unit  or  time).  In  an  apportionment  we 
start  with  a  requirement,  which  is  converted  to  total  failures 
to  be  permitted  per  unit  of  time.  We  then  allocate  to  the 
various  subsystems  a  share  of  the  failures  to  be  permitted.  The 
apport ionment  in  no  sense  indicates  that  the  particular  level  of 
reliability  required  can  be  achieved.  It  merely  says  that  if 
the  apportioned  values  ar«  achieved,  the  system  .ill  meet  its 
requ i r ement s . 

To  make  the  apportioned  values  of  reliability  realistic,  con¬ 
sideration  must  be  paid  to  the  factors  mi kino  reliability  dif¬ 
ficult  or  expensive  to  achieve.  The  development  of  high  relia¬ 
bility  is  cost  lv.  F.stabl  ishinu  requirements  higher  than  neces¬ 
sary  is  uneconomical.  Apport ionmt nt  techniques  should  be  based 
on  the  factors  that  define  the  relative  effort  and  cost  of 
achieving  the  required  reliability  for  the  system. 

The  ar;  art ionment  of  system  reliability  involves  solving  the 
basic  equality 

MR.  ,  K-.  .  .  .  . R„  )  -  R  (6-1) 


where  . 

R.  is  the  apportioned  reliability  parameter  hr  the  ith 
an  it  , 

R  is  t'b  e  system  reliability  requirement  parameter ,  and 
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f  is  the  functional  relationship  between  unit  and  system 
reliability. 


For  a  simple  series  system  in  which  the  R.  (t)s  represent  pro¬ 
bability  of  survival  for  t  hours,  Equation  6-1  becomes 


R,  (t)  •  Rr  (t)  .  .  . R.  (t)  -  R  ( t ) 


(6-2) 


Theorem ’ rally ,  Equation  6-2  has  an  infinite  number  of  solutions, 
assuming  no  restrictions  on  the  apportionment .  The  problem  is 
to  establish  a  procedure  that  yields  a  unique  or  limited  number 
of  solutions,  b^  which  consistent  and  reasonable  reliabilities 
may  be  allocated. 


The  program  and  mettr  ds  presented  in  this  chapter  can  apply  to 
the  sub-allocation  or  reliability  within  the  various  units.  The 
apportionment  program  is  necessarily  one  of  continual  refinement. 
Original  requirements  determined  at  the  design  stage  should  be 
critically  examined  and  revised  as  more  experience,  knowledge, 
and  test  data  became  available  during  the  advance  of  the  system 
life-cycle  through  the  design,  development,  and  production 
phases . 


1.  APPORTIONMENT  OF  INHERENT  RELIABILITY 


1.1  PAS IC  THEORY 

In  Chapter  foui  we  developed  the  statement  of  reliability  in 
terms  of  probability  of  success  based  on  a  failure  density 
function.  A  failure  density  function  can  be  determined  for  any 
set  of  equipment. 

It  has  been  shown  that  the  probability  of  success  (the  reliabil¬ 
ity)  of  a  system  is  the  probability  that  no  failure  occurs  during 
the  time  in  question.  In  the  general  case  the  reliability  may 
be  stated 


R,  *  exp 


t- 


t, 


F,  (x)  dx 


J 


(6-3) 


where  the  F( (x)  is  the  instantaneous  hazard  function.  Where  the 
distribution  function  is  exponential,  this  hazard  function  is 
the  "ccnstec.t"  failure  rate  V.  In  the  norma1  case,  it  is  the 
ordinate  of  the  normal  curve 


n  ( a) 


1 


-  4  <^)J 
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In  conducting  an  apportionment,  we  start  with  defining  our  system 
as  a  series  of  units  so  tnat  equation  6-2  applies.  Where  equip¬ 
ments  are  duplicated  in  parallel . .  it  is  necessary  to  treat  the 
combination  as  a  single  unit..  Applying  equation  6.3 

t 

...  -  ^  F„ (x) dx  1 


If  we  set  Rj  -  Ra'  and  solve  for  the  a5 
Ft  (x) dx 


^  -  jjj  Fi  (x)dx 
i=l 

factors  have  the  following  characteristics: 

n 

V' 

L  ai  =  1 

i=l 

(b)  Each  at is  the  fraction  that  represents  that  portion  of 
the  total  probability  of  failure  in  the  system  attributable  to 
the  ith  unit. 

We  can  apportion  the  reliability  by  selecting  factors  for  each 
unit  (the  at )  such  that  (a)  the  sum  of  the  factors  add  to  1  and 
(b)  each  factor  is  the  fraction  of  the  failures  allowed  for  the 
system  to  be  permitted  to  the  unit. 

1.2  SELECTION  CRITERIA 


The  ideal  apportionment  would  be  that  allocation  of  requirements 
resulting  in  the  most  economical  use  of  resources,  including 
time  and  cost.  Among  others,  the  following  considerations  should 
be  considered. 

(a)  The  complexity  of  the  system  will  have  an  effect  on  the 
achievable  reliability.  The  more  complex  the  system  is,  the 
greater  the  number  of  subassemblies  and  modules,  the  more  dif¬ 
ficult  and  costly  it  is  to  achieve  a  high  reliability.  Imposing 
an  unrealistically  high  reliability  on  the  more  complex  systems 
increases  the  cost  disproportionately  when  compared  with  the 


the  aj 
(a) 


R  =  exp 


^  Fi  (x)dx  -  ^  F7  (x)dx 


v 


f 


=  exp  [  ^  ‘  Fj  (x)  dx ] 


L=1 
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effect  of  increasing  the  reliability  requirement  fcr  simpler 
sys  k.ems , 

t b )  The  amount  of  development  and  research  required  to  pro  • 
duce  the  system  will  greatly  influence  the  time  and  cost  of  de¬ 
velopment.  Imposition  of  a  high  reliability  requirement  on  a 
system  under  development  will  increase  the  development  time, 
numbers  of  tests  required  to  obtain  the  reliability  and  the  cost. 
Equipments  considered  present  "State  of  the  Art"  are  penalized 
less  by  high  reliability  requirements. 

(c)  The  intended  operational  environment  will  have  an  effect 
''n  the  achievable  reliability.  A  system  to  be  used  in  a  "rugged" 
environment  will  tend  to  cost  more  to  develop  to  an  equal  relia¬ 
bility  than  a  similar  one  to  be  used  under  less  severe  conditions, 

(d)  The  length  of  time  the  equipment  is  required  to  perform 
will  influence  the  achievable  reliaoility.  It  will  require  more 
development  effort  and  cost  to  produce  a  system  capable  of 
operating  for  a  long  period  of  time  without  failure  than  to 
develop  one  for  a  shorter  period  of  use. 

(e)  The  need  for  high  reliability  in  a  system  is  based  on  the 
importance  of  its  operation.  A  system  whose  failure  would  not 
jeapordize  the  accomplishment  of  the  mission  need  not  be  highly 
reliable.  To  the  extent  that  failures  can  be  tolerated,  lower 
reliability  requirenu  its  should  be  imposed. 

Apportionment  of  reliability  is  a  trade-off  between  the  relia¬ 
bilities  of  units  to  achieve  a  specified  system  reliability. 

By  imposing  high  reliability  requirements  on  those  units  in  which 
high  reliability  is  easier  to  attain,  and  lower  requirements  on 
those  in  which  high  reliability  is  more  difficult  and  more 
costly,  the  overal  ..  cost  of  the  system  development  mav  be 
reduced. 

Numerous  methods  have  been  used  to  select  the  factors  for  the 
opportionmont  to  achieve  this  cost  (and  time)  improvement. 

2.  TECHNIQUES  OF  APPORTIONMENT 

2.1  EQUAL  APPORTIONMENT 

In  the  absence  of  any  definitive  information  on  the  system, 
other  than  the  fact  that  a  subsystems  will  be  used,  the  only 
rational  basis  to  use  wouicj  be  equality.  If  each  a,  is  set  at 
1/N,  the  two  requirements  are  met.  Each  subsystem  is  then 
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required  to  have  a  r 


1/N  H. 

el  lability  of  (R)  or  ./. 


R,  The  product  of 


N  N 

the  N  system  reliabilities  is  then  {  R)‘  -  R, 


2./  CONSIDERATIONS  Of  IMPORTANCE  AND  COMPLEXITY 

Task  group  2  of  the  AGREE  Study  (i)  recommends  an  apportionment 
for  electronics  systems  based  on  the  importance  of  the  unit  and 
its  complexity.  The  exponential  distribution  of  times  to 
failure  is  assumed  to  apply.  Let  a  system  consist  of  k  units. 

For  i  -  1,  2  ..  k  let 

ntj  =  MTBF  (mean  life)  of  the  Lth  unit. 

t,  =  Operating  time  during  the  mission  required  of  the  ith 
unit . 

Wj  =  Probability  that  the  system  will  fail,  given  that  the 
ith  unit  fails  (importance  factor). 

rM  =  number  of  modules  (e.g.,  tubes)  in  the  ith  unit. 

£ 

h7  =  Total  number  of  modules  in  the  system  =  t  n, 

i=l 

It  is  desired  to  apportion  the  reliability  between  the  units  .in 
such  a  manner  that  each  module  make  an  equal  contribution  to 
mission  success.  The  mean  life  to  be  required  for  each  equip¬ 
ment  is  computed  from  the  formula 

mt  =  - -  (6-5) 

(J1-)  f-ln  R) 

N 

Example:  For  a  system  reliability  requirement  of  R- .90 (-LnR= . 103 ) 


n 

w 

t 

m 

RCVR 

20 

.7 

4  hrs 

402 

XMTR 

30 

.5 

4  hrs 

218 

RADAR 

200 

.8 

4  hrs 

52 

IFF 

50 

.2 

4  hrs 

52 

N 

300 

The  equation  for  the  reliability  of  the  ith  unit  is: 
R,  =  r 


l 
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j 

|  We  can  rewrite  6-5  to  show: 

i  , 

!  t./m.  -  (Jp)  (— )  {-In  R' 

i  N  w , 


So, 


,  n .  l_ 
.  (R)  "  ’ 


T’us  equation  shows  that  the  basis  of  the  factors  a.  in  equation 
6-4  are  made  up  of  the  product  of  numbers  representing  the  re¬ 
lative  complexity  of  the  equipment  and  numbers  representing  the 
importance  of  the  unit  to  mission  success. 


2.3  FURTHER  EXTENSION  TO  MEGHAN  T'CAL-ELECTR  ICAL  SYSTEMS 


The  Bceinci  Company,  in  its  Reliability  Manual  (2)  proposes  an 
alternate  method  of  selecting  the  factors  of  apportionment.  The 
parameters  to  be  considered  are: 

(a)  System  Complexity:  Complexity  is  evaluated  by  consider¬ 
ing  the  probable  quantity  of  parts  or  components  making  up  the 
system,  and  is  also  judged  by  the  assembled  intricacy  of  these 
parts  or  components.  The  least  complex  system  is  rated  at  1. 

The  system  considered  highly  complex  is  rated  at  10. 


(b)  State  of  the  Art:  The  state  of  present  encrineering  pro- 
uress  in  all  fields  is  considered.  The  system  least  developed 
is  assigned  a  value  of  10  and  the  system  most  highly  developed 
is  assigned  a  value  of  1.  All  other  systems  are  evaluated  be¬ 
tween  10  and  1. 

(c)  Performance  Time:  The  system  that  operates  for  the 
entire  mission  time  is  rated  10,  and  the  system  that  operates 
a  minimum  time  during  the  mission  is  rated  at  1.  All  other 
systems  arc  evaluated  between  these  two  extremes. 

(d!  Env i ronmenta  1  Cond i 1 1 ons :  Environmental  conditions  can 
also  be  rated  from  10  through  1.  Systems  expected  to  experience 
harsh  and  extreme  conditions  during  performance  will  be  classi¬ 
fied  as  10  and  systems  expecting  to  encounter  the  least  severe 
conditions  will  be  classified  as  1.  All  other  systems  shall  lie 
between  these  two  extremes. 

A  typical  computation  is  shown  in  Figure  o  ae  selection  of 

the  factors  is  done  by  engineering  judgment  based  on  the  engineers 


* 

i. 

■■  -a,  J»lf  TUI 
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predictions  of  the  relative  effect  of  various  factors  on  the 
reliability  of  performance  of  the  system.  Using  a  philosophy 
that  there  are  no  new  componen :  s ,  new  systems  are  rearrangements 
of  known  components.  The  engineer  would  judge  what  types  of 
parts  and  components  would  be  used  in  a  new  system  and  what 
effect  the  expected  use  of  these  parts  would  have  on  the  relia¬ 
bility  of  the  parts.  Where  particular  components  had,  in  his 
experience,  been  unreliable  in  a  particular  environment,  he 
would  reflect  this  in  his  choice  of  factors.  Factors  may  be 
selected  by  individual  engineers  or  through  some  form  of  voting 
technique  as  describe'  in  paragraph  3. 

2.4  EXTENDED  METHOD  FOR  ELECTRONICS  SYSTEMS 


A  further  development  of  the  AGREE  system  (2.2)  has  been  made  by 
Arinc  Corporation  (3). 

2.4.1  Elements  Considered  in  the  Apportionment: 

(a)  Unit  Essentiality:  The  concept  of  essentiality,  used  to 
describe  the  effect  of  unit  failure  on  mission  success,  is  con¬ 
sidered  unity  if  a  failed  unit  does  not  have  a  functional  dup¬ 
licate.  It  is  defined  as  follows: 

The  essentiality  of  a  unit  is  the  probability  that 
the  system  will  fail  to  accomplish  its  mission  if 
the  unit  fails  while  all  other  units  perform  satis- 
factor i ly . 

At  the  design  stage  of  system  development,  the  likelihood  is 
that  the  essentiality  of  various  units  within  the  system  will 
have  to  be  assigned  intuitively,  on  the  basis  of  experience 
gained  with  similar  systems.  If  appropriate  system  failure  data 
is  available,  essentiality  can  oe  estimated  by  the  ratio, 

g  -  Nvmber  of  mission  failures  due  only  to  ith  unit  failure 
1  Number  of  ith  unit  failures 

(b)  Basic  Failure  Data:  The  allocation  procedure  is  based 
on  the  relative  reliabil ’  «-ies  to  be  expected  of  various  units  of 
a  system,  as  determined  from  past  experience.  The  electronic 
functional  levels  to  which  this  procedure  is  applicable  corres¬ 
pond  to  the  functions  performed  by  individual  element  groups 
(AEG's).  An  active  element  group  is  defined  as  consisting  of  an 
active  element  (o>  e  part,  such  as  a  tube,  capable  of  performing 
valving  or  controlling  action)  plus  the  associated  passive  parts; 
examples  of  active  element  groups  include  amplifiers,  oscillators, 
mixers,  and  rectifiers.  Equivalents  are  provided  for  non-elec- 
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tronic  components. 

2.4.2  Procedure  for  Reliability  Apportionment:  The  worksheet 
used  for  conducting  an  apportionment  is  given  in  Figure  6-11. 

Steps  in  completing  the  worksheet  are: 

(a)  Identify  the  units, 

(b)  Estimate  the  essentiality  index  (paragraph  2.4.1b),  E, 

(c)  Record  or  estimate  that  portion  of  the  system  operating 
time  the  unit  will  be  required  to  operate,  t, 

(d)  Develop  unit  failure  indices,  K, ,  based  on  class  of 
equipment,  relative  failure  rates  for  the  class  and 
number  of  modules  of  the  class  in  the  unit  (refer  to 
reference  3). 


(e)  Compute  the  fa : lure  index  ratio 


/  K, 

1=  1 


(f)  Compute  the  allocated  unit  reliabilities  from  the 
equation 


Ri  *  1  - 


1  -  (R)W‘ 

E, 


where  R,  is  the  desired  reliability  apportioned  to  the 
ith  unit, 

R  is  the  system  required  reliability. 

Figure  6-12  shows  a  typical  computation  of  the  unit  failure 
indices.  The  system  is  a  bombs ight  consisting  of  three  units, 
power  supply,  navigation  computer  and  optical  equipment .  The 
unit;:  ai  e  considered  in  modified  series,  since  both  the  power 
supply  and  optical  equipment  must  work.  In  the  event  the  navi¬ 
gation  computer  should  fail  the  optical  equipment  can  be  con¬ 
trolled  manually.  The  essentiality  of  the  power  supply  and 
optical  equipment  are  unity.  On  the  basis  of  performance  of 
similar  systems,  estimates  were  made  that  for  every  100  missions 
in  which  the  navigation  computer  failed,  57  mission  failures 
resulted.  So  the  essentiality  of  the  navigation  computer  was 
estimated  at  57/100  =  .57. 


RELIABILITY  ALLOCATION  WORKSHEET 
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Proceeding , 
Step  1: 

Step  2  : 

Step  3  : 

Step  4: 


Step  5: 

Step  6: 

Step  7  : 


The  functional  category  column  is  divided  into 
electronic  and  non-electronic  groups. 

The  relative  failure  rates  (K,  or  K ' )  for  each 
functional  category  is  entered  in  the  appropriate 
column . 

The  number  of  estimated  AEG ' s  of  each  category  within 
each  unit  is  entered  in  the  column  headed  f. . (j=l,2,3) 
and  the  electronic  category  rows  are  summed  to  obtain 
the  entries  in  the  column  headed  f, . 

The  average  electronic  failure  index  is  computed  in 
the  following  manner: 

(a)  Form  the  total  unadjusted  electronic  failure 
index 

K  =  40(4.3)  ^  10(3.0)  +  230(3.0)  =  892 

e 

(b)  Determine  the  number  of  electronic  AEG's  in 
group  (a) . 

F  =  40  +•  10  +  2  30  =  280. 
e 

(c)  Form  the  average  electronic  failure  index: 

K  -  892/280  •=  3.186. 
e 

Convert  each  k,'  to  a  failure  rate  relative  to  the 
electronics  group  by  multiplying  the  relative  failure 
rates  by  K  -  3.186.  Enter  in  the  k,  column. 

Adjusted  relative  failure  rates,  k,  , 

(ax  Transfer  the  k.'s  to  the  appropriate  unit  column; 

(b)  k,  and  kB  rcsr.ai.n  unaltered  for  Group  (a),  but 
since  k,  in  Unit  2  has  a  transistor  active 
element. ,  using  an  adjustment  factor  of  0.3, 
compute  k,--  (0.3)k-,  -  0.9. 

Unit  failure  indices 

T~ 

Using  the  formula  K.  =  i.  , k .  for  the 
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failure  index  of  the  jth  unit,  compute  k. ,  k  and  k 

Step  8:  The  values  far  the  unit  failure  indices  are  entered 
in  the  allocation  worksheet.  Figure  6-lr’ 


Step  9:  Entering  the  allocation  worksheet  with  the  failure 
indices,  compute  the  failure  index  ratio 


w.  = 


i  =  1 

Step  10:  The  reliability  apportionment  R,  is  computed  for 

w, 


each  unit  using  the  formula  R.  =  1  - 


1-R 


E, 


2.5  ALTERNATE  BOEING  METHOD 


For  an  alternate  method  cf  selecting  the  apportionment,  the 
following  approach  (4)  is  proposed  by  F.  E.  Marsh,  tin  Boeim 
Company.  Given  a  reliability  goal,  R,  for  an  item  comprising  n 
units  in  series  and  assuming  an  exponential  distribution  of 
times  to  failure,  the  reliability  goal,  R, ,  apportioned  to  unit 
i  is  : 


where  r 


where . 


,  ,  w, 
R,  =  (R)  ! 


w, 


and 


a, 


I  (I  +  I,  + 

UK  f 


I  ! 
m 


I  =  Index  of  state  of  the  art,  computed  usino 
u 

engineering  data  on  system  reliability 
growth  rates; 


1^  -  Index  of  complexity,  computed  to  account 
for  relative  complexity  and  j  edundancy  ot 
the  unit; 


1^  =  Index  cf  environment,  computed  from  esti¬ 
mates  of  unit  stress  levels  due  to  environ¬ 
mental  co..Ji t  ions  ,  and 


I  *  Index  of  operating  time,  cn  ..put  ed  from  the 
operating  time  of  the  unit  and  the  opera¬ 
ting  time  for  the  system. 
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2.6  USE  OF  COST  OF  AC  H I  El7  EK  ENT 


It  is  reasonable  to  assume  that  if  a  contract  for  a  system  is  let 
with  no  reliability  requirement,  a  system  built  in  accordance 
with  standard  design  practices  will  result.  If  a  reliability 
requirement  is  imposed,  the  system  will  cost  more  by  virtue  of 
the  fact  that  additional  effort  is  required  by  the  contractor. 

How  much  more  the  system  will  cost  will  depend  upon  two  basic 
factors . 


(a)  By  what  degree  the  reliability  requirement  exceeds  that 
expected  (that,  experienced  using  standard  design)  . 

*b)  The  complexity  of  the  system  contracted  for. 

Bird  Engineering  Research  Associates,  Inc-  (8)  found  in  past  pro¬ 
curements  a  relationship  between  the  cost  and  reliability  of 
equipments  and  their  relative  complexity. 

-1.39 

m  -  187,000  N 

QQ 

C  =  (.891)  (29698)N 

Where  m  was  the  achieved  MTBF,  c  the  contract  cost  anc  N  the 
number  of  active  element  groups  as  defined  by  MIL  STD  756.  The 
constants  appxy  to  shipboard  equipment. 

For  an  improved  product,  experience  (Chapter  26)  indicates  a 
.  el  atior.ship  between  cost  and  reliability  of  the  form 

c'  -  C  =  C  in  (-)0*3  (6-7 ) 

n\ 

Where  the  prime  is  used  to  distinguish  between  an  equipment  pro¬ 
duced  under  a  different  level  oi  j.eliability  effort.  If  we 
assume  a  particular  equirement,  R,  for  the  reliability  of  the 
complete  system,  this  is  achievable  by  any  combination  of  sub¬ 
system  reliabilities,  R5  that  satisfy  the  relationship. 


(6-6) 


k 

r  =  n  r, 
i=  1 


For  a  system  consisting  of  three  serial  subsystems  R  =  RjX  R?  x  R3 , 
where  the  subscripts  refer  to  the  subsystems 


-  ( tt  /m*  +  tj,  /m2'  +  t3  /m3' ) 


e 
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assuming  an  exponential  distribution  of  times  to  failure.  The 
exponent  must  satisfy  the  relationship  3 

V 

-  In  R  -  t.x  /m'  +  tg/rnj  +  t3/ro3'  =  L  tj/m,'  (6-8) 

i=l 

As  may  be  seen  from  Figure  6-1S,  the  selection  of  a  particular 
set  of  requirements  for  reliability  imposes  a  particular  cost  on 
the  development  program.  A  higher  reliability  requirement  for 
any  system  tends  to  increase  the  cost  for  that  system.  Lowering 
the  requirement  should  reduce  the  cost.  Those  in  which  the  in¬ 
cremental  cost  of  improvement  is  greatest  should  be  given  lesser 
requirements,  increasing  the  requirements  correspondingly  for 
those  with  lesser  incremental  costs.  This  can  be  done  as 
fcxlows i 


The  total  cost  of  the  program  would  be  the  sum  of  costs  of 
developing  the  individual  systems  with  the  conventional  program 
costs  plus  the  additional  cost  for  each  Systran  necessary  to 
improve  the  reliability  to  achieve  the  system  reliability,  R« 

We  can  define  this  additional  coat, 


*CR  = 


u  ln(~) 

R»i 


'0.2 


+  C2  In  ) 
nu, 


C.S 


+  CjlnfjJ3) 
ro3 


o.a 


(6-9) 


To  obtain  the  minimum  cost  that  will  achieve  the  required  /system 
reliability,  we  can  differentiate  the  equation  below  and  set  it 
equal  to  zero. 

f  t,'  o.3  r 

4C„  -  u  c,  e  )  -  x  C In  R  -  L  (r1.)  ) 

R  i=1  ».  t.x  “i 


where  the  last  term  is  the  constraint  imposed  by  equation  6-8 


dm, 


2C, 


m, 


+  X  — Vr3  *  0  for  each  i 

(i«i) 


that  is  — V  " 

m,  X 


This  says  that  the  ratios  of  the  ~x  should  be  proportional  to 

the  C, . 

*88 

Since  the  Ct  are  related  to  the  complexity  by  the  ratio  H* 
(Equation  6-6)  the  optimum  solution  to  the  apportionment  of 
reliability  would  be  achieved  by  selecting  as  complexity  factors 
the  AEGs  raised  to  the  .88  power. 
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Reference  (8)  provides  a  different  correlation  between  initial 
cost  and  complexity  to  be  used  in  case  of  developments  in  which 
performance  involves  design  beyond  the  conventional  state  of  the 
art,  that  is,  for  such  cases 

88 

C  =  1.464  (29698)N* 


In  an  apportionment,  to  account  for  the  additional  effort  to 
achieve  major  advances  in  the  state  of  the  art  in  the  design 
the  complexity  factor  N*®®  should  be  multiplied  by  a  factor  of 


1.464 

.891 


1.644. 


Figure  6-20  gives  an  example  of  the  application  of  this  method. 
Subsystems  C  and  E  are  assumed  to  require  major  advances  beyond 
the  present  state  of  the  art  in  development.  Subsystems  A,  B 
and  D  are  conventional  design  with  minimum  acceptable  reliability 
requirements  established  somewhat  beyond  present  normal  achieve¬ 
ments  . 


3.  VOTING  TECHNIQUES 

In  the  early  stages  of  development  of  a  system,  very  little  may 
be  known  about  the  hardware.  Each  of  the  techniques  in  para¬ 
graph  3  requires  the  application  of  more  or  less  judgment  in 
selecting  some  of  the  factors.  If,  as  in  the  method  covered  in 
paragraph  2.3,  the  indices  assigned  are  not  representative  of 
the  ultimate  equipment,  the  apportionment  will  create  more  pro¬ 
blems  than  it  will  solve.  Recognizing  this,  recent  methods  of 
apportionment  attempt  to  limit  the  aaount  of  judgment  that  must 
be  applied.  But  no  method  can  eliminate  the  *  juirement  entirely. 
In  conducting  an  apportionment,  then,  the  task  is  to  (a)  select 
the  method  apparently  most  appropriate  to  the  problem  considering 
the  details  known,  the  nature  of  the  equipment  and  the  availa¬ 
bility  of  pertinent  data;  (b)  Identify  the  area*',  iu  which  judg¬ 
ment  is  required,  and  (c)  Arrange  to  obtain  the  best  possible 
responses  from  qualified  individuals. 

3.1  FRAMING  THE  QUESTIONS 


As  was  developed  in  paragraph  2,  the  apportionment  depends  on 
the  selection  of  factors  that  are  proportional  to  the  number  of 
failure  to  be  “permitted"  to  the  unit.  These  factors  should  be 
so  selected  as  to  minimize  the  difficulty  of  system  development. 
That  is  a  comparison  must  be  made  on  (a)  amount  of  development 
required  for  the  unit;  (b)  complexity  of  the  unit;  (c)  expected 
effect  of  the  planned  operational  use  on  the  difficulty  in  dev- 
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elopment;  (d)  the  need  for  high  unit  reliability  to  achieve  high 
system  reliability. 

Any  general  question  posed  to  the  judges  such  as  "list  these 
equipments  in  the  ascending  order  of  expected  failure  rates  (or 
descending  order  of  MTBF) "  will  not  yield  much  valid  information. 
Consider  a  more  detailed  set  of  questions,  such  as 

What  is  the  level  of  vibration  you  expect  this  equipment 
to  be  subjected  to? 

What  level  of  vibration  does  equipment  of  this  type 
normally  withstand? 

Do  you  think  this  difference  will  cause  you  to  have 

(a)  fewer  failures? 

(b)  more  failures? 

(c)  no  difference  in  the  number  of  failures? 

These  latter  questions  forces  the  judge  to  concentrate  upon  one 
effect  and  provide  his  best  judgment  in  an  area  in  which  he 
might  feel  confident. 

The  questions  then,  should  be  framed  in  a  way  to  relate  to  the 
experience  of  the  judge  and  should  provide  a  suggestion  as  to 
how  to  go  about  arriving  at  a  decision. 

With  a  large  number  of  interrelated  factors,  each  factor  must 
be  given  an  appropriate  weight.  The  difficulty  of  making  a 
judgment  involving  many  factors  tends  to  make  such  judgments 
somewhat  erratic  and  ineffective.  In  framing  the  questions, 
they  should  then  be  limited  in  the  factors  that  are  to  be  con¬ 
sidered,  and  the  factors  should  be  within  an  area  in  which  the 
judge  feels  competent. 

3.2  SELECTING  THE  JUDGES 


The  less  there  is  known  about  the  unit,  the  greater  is  the 
importance  of  utilizing  knowledge  of  engineers  competent  in  the 
field.  When  a  designer  designs  a  new  system  he  doesn't  reinvent 
the  components  or  the  circuits.  A  bearing  in  a  motor  is  the  same 
a3  bearings  in  other  motors.  The  new  assembly  has  some  innova¬ 
tions,  but  also  it  has  many  parts  used  in  well  known  ways. 

Before  the  equipment  is  designed,  a  well  qualified  designer  can 
tell  you  the  characteristics  of  the  parts  he  would  use.  He 
would  know  a  great  deal  about  the  reliability  of  those  parts. 
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In  attempting  to  obtain  an  estimate  or  judgment  on  a  particular 
factor ,  there  is  a  real  need  to  assure  that  the  source  is  com¬ 
petent  in  the  area  of  interest. 

3.3  METHOD  OF  FAIRED  COMPARISONS 

3.3.1  Conducting  the  Survey;  If  the  question  to  be  resolved 

is  very  complex,  such  as  the  relative  amount  of  development  test¬ 
ing  for  the  various  units  required  to  develop  the  system,  it  may 
not  be  possible  for  an  engineer  or  a  committee  to  set  relative 
values.  However,  it  should  be  possible  for  the  engineer  to 
make  judgments  of  less  complexity,  say  between  two  of  them.  A 
method  due  to  Thurstone  &  Mosteller  (5}  has  been  devised  to  use 
such  comparison  of  pairs  to  evolve  a  relative  ranking  of  the 
item  of  interest. 

Example:  A  new  ship  class  is  being  developed,  the  major  systems 
required  for  the  "special"  mission  are: 

A.  "Star  Tracker"  Navigation. 

B.  "Ship  to  Space"  UHF  wide  channel  communications. 

C.  "Lock-On"  tracking  system. 

D.  Data  acquisition  and  storage  system. 

E.  Computer  analyzer. 

F.  Data  display  system. 

The  item  of  interest  is  the  relative  cost  of  development  of  the 
subsystems.  It  was  decided  to  obtain  the  judgments  of  eight 
engineers  who  had  been  working  on  radar,  communications  and  com¬ 
puter  complexes.  The  parameters  of  performance  of  each  sub¬ 
system  could  be  defined.  The  question  was  framed.  "In  the  de¬ 
velopment  of  these  two  subsystems,  E&F,say,  which  do  you  believe 
will  require  the  most  developmental  testing  to  provide  an  oper¬ 
ational  system?" 

Eight  identical  sets  of  cards  were  made,  each  set  containing  the 
comparison  between  each  of  the  pairs  of  systems  (AB,  AC,  BC  etc.). 
Each  engineer  selected  completed  and  returned  his  set. 

3.3.2  Analysis  of  results: 

(a)  The  individual  cards  were  scored  as  follows.  If  the  box 

marked  "moderate"  was  checked,  the  card  scored  1  "consider¬ 
ably  more"  was  similarly  counted  2.  If  the  system  first  in 
alphabetical  sequence  was  checked,  the  card  was  scored  +, 
otherwise  -.  A  sample  card  is  shown  in  Figure  6-23. 


SAMPLE  CARD  PAIRED  COMPARISON  METHOD 


Engineer 
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(b)  The  analysis  was  conducted  in  two  ways.  The  averages  of 
the  car  1  were  recorded  in  a  matrix,  Figure  6-25.  It  was 
noted  that  a  definite  order  was  indicated.  C  was  felt  to 
require  more  development  than  any  other  system,  A  next  and 
the  remainder  following  in  the  order  B,  E,  F,  D.  These 
were  replotted  in  matrix  form,  Figure  6-26. 

Reasoning  that  the  comparisons  would  be  more  meaningful  between 
those  considered  close  together  the  "strong"  diagonal  (C  to  A, 

A  to  B,  B  to  E,  etc.)  was  selected  as  the  best  relative  compar¬ 
ison.  Setting  the  one  requiring  the  least  development  testing 
as  the  Standard,  D=l.  The  relative  scale  of  test  requirements 
came  out  as  follows: 


System 


Relative  test  requirements 


D 

F 

E 

B 

A 

C 


1.00 

1.125 
2.000 
2.2  50 
3.000 

3.125 


An  attempt  was  made  to  improve  the  analysis,  using  more  of  the 
information  obtained,  followino  the  analysis  described  by 
reference  7. 

Having  the  preference  matrix  the  preferences  were  normalized 

X+  2 

using  the  equation  X.  =  — (Figure  6-27). 

4 

The  deviates  were  computed  from  the  relationship 

,  .  1  1  '  -XV  2 

(X.  )  .  .  =  -tt—  e  dx 

The  deviate  matrix  is  shown  in  figure  6-28.  The  deviates  con¬ 
tained  in  the  elements  correspond  to  areas  under  the  normal 
curve.  The  average  of  each  row  was  computed  and  tabulated  in 
the  column  r. .  The  difference  between  the  average  deviates  in 
•ach  row  were  computed,  using  the  relationship 


r . 


X‘  /  ■ 


dx 


The  values  T  then  are  used  as  proportional  to  the  level  of 
testing  required  in  a  program  to  develop  the  systems  to  an 
operational  condition. 


t 
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4  *  USE  OF  APPORTIONMENT  TECHNIQUES 

4.1  CONCEPTUAL-  PHASE 


Apportionment  in  the  conceptual  phase  is  primarily  for  determin¬ 
ation  of  feasibility.  The  question  that  must  be  decided  is  the 
element  of  risk  involved  in  undertaking  the  development.  The 
apportionment  assists  in  this  determination  by  setting  reliabil¬ 
ity  goals  against  which  to  measure  the  capability  of  the  indus¬ 
try.  The  AGREE  method  (paragraph  2.2)  for  electronics  systems 
and  the  Boeing  method  (paragraph  2.3)  for  mechanical  systems 
are  appropriate.  For  electronics  systems,  the  cost  evaluation 
corrective  factors  may  be  applied  (paragraph  2.5)  to  evaluate 
cost  consequences . 

4.2  PRELIMINARY  DESIGN  PHASE 

In  the  early  phases  of  design,  the  purposes  for  an  apportionment 
are  to  provide  requirements  for  supplier  and  contractor  furn¬ 
ished  systems,  and  to  set  targets  (requirements)  to  be  achieved 
in  the  design  per  formed  by  the  prime  contractor  --  or  internally 
within  the  Bureau.  The  apportionment,  to  be  comparable  with 
design  predictions,  must  be  formulated  on  the  same  basis  as  the 
predictions  will  later  be.  It  should  include  a  comparable 
statement  of  environment  and  operating  time.  For  electronic* 
systems,  the  more  detailed  considerations  of  the  Arinc  method 
(paragraph  2.4>  should  be  used.  For  mechanical  systems,  the 
Boeing  method  (paragraph  2.3)  is  the  only  useful  method  known. 

In  attempting  to  use  this  system,  the  selection  of  weighting 
factors  must  be  developed  in  such  a  way  that  they  reflect  the 
effect  of  the  particular  factor  on  the  failure  rate  that  will 
be  achieved  when  the  equipment  becomes  operational. 

4.3  EVALUATION  OF  CONTRACTORS  APPORTIONMENT 


When  contractors  perform  an  apportionment  to  allocate  a  system 
requirements  to  units,  the  Bureau  engineer  responsible  must 
evaluate  his  apportionment  process  to  assure  that  the  unit  re¬ 
quirements  are  based  on  a  sound  appraisal  of  cost  and  effective¬ 
ness.  The  techniques  in  paragraph  2.0  demonstrate  the  methods 
most  likely  to  be  used.  Where  some  other  method  is  used,  the 
basis  of  the  method  should  be  evaluated  against  the  criteria 
(equation  6-4  and  paragraph  1.2). 

4.4  SUMMARY 


In  summary,  reliability  apportionments  are  made: 
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(a)  To  set  reliability  requirements  for  units  of  a  system 
to  establish  procurement  and/or  design  objectives. 

(b)  To  provide  a  means  of  measuring  progress  toward  achieve¬ 
ment  of  the  system  reliability  objective. 

The  value  of  the  apportionment  in  achieving  these  objectives 
depends  or.  the  care  and  judgment  used  in  making  the  apportion¬ 
ment  . 


Since  the  apportionment  is  used  primarily  as  a  guide  to  the 
achievement  of  the  system  objective  it  should  be  continuously 
updated  as  the  design  progresses  ana  used  to  modify  the  require¬ 
ments  imposed  on  the  component  suppliers  and  subsystem  designers 
as  more  information  becomes  available.  Apportionment  should  be 
continuously  used  as  a  tool  to  achieve  the  system  objective. 
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CHAPTER  7 

STRESS-STRENGTH  ANALYSIS 


The  concept  of  safety  margins  (1)  has  been  developed  from  the 
traditional  safety  factor  of  the  structural  design  disciplines. 
Safety  factors  in  design  have  long  been  used  with  a  high  degree 
of  success  based  on  knowledge  evaluated  from  successful  appli¬ 
cations,  simple  testing,  or  proofing.  They  are  predominately 
empirical  in  nature  and  are  usually  intuitive,  based  on  engineer¬ 
ing  judgment.  Safety  factors  are  traditionally  generous  and  may 
often  cause  weight  and  cost  penalties  which  cannot  be  tolerated. 
Safety  margins  are  essentially  modified  safety  factors  and  are 
derived  from  comparing  a  distribution  of  possible  loads  to  a 
distribution  of  possible  resistive  strengths. 

No  two  things  can  be  identical;  they  are  inherent iv  variable  to 
some  degree.  The  variation  in  material  from  lot  to  lot  and  from 
producer  to  producer  is  well  known.  The  variation  in  loads  from 
experiment  to  experiment  and  between  periods  or  cycles  of  use  can 
equally  be  established.  As  discussed  in  Chapter  9,  stresses  and 
loads  can  be  described  by  distribution  functions  in  which  the 
frequency  of  occurrence  of  stresses  or  strengths  is  compared  to 
the  stresses  or  strengths  occurring. 

The  concept  is  not  limited  to  the  structural  field.  The  general¬ 
ization  of  the  stress-strength  analysis  to  electric,  hydraulic 
or  mechanical  equipment  is  obvious  wherever  a  (generalized)  stress 
exceeds  the  strength  of  the  material  to  resist  a  failure  result. 

If  the  stresses  and  strengths  vary  in  an  identifiable  fashion,* 
the  frequency  with  which  failures  can  be  expected  to  occur  can 
be  computed  by  the  stress-strength  technique. 

1.  THE  PRINCIPLE  OF  SAFETY  MARGINS 


It  would  be  ideal  to  have  specifications  which  would  increase 
both  reliability  and  performance.  We  may  come  closer  to  achieving 
this  goal  by  replacing  the  principle  of  rigidly  specified  safety 
factors  by  the  more  effective  principle  of  safety  margins  to  take 
account  of  the  fact  that  unreliability  is  caused  not  only  by  low 
averages  but  also  by  large  variations  of  strength. 

Vai  iations  may  be  large  or  small,  as  illustrated  in  Figure  7-3. 
Although  components  A  and  B  have  the  same  average  strength, 
component.  B  evidently  is  less  consistent  than  conponent  A.  It 
is,  therefore,  imperative  that  the  characteristic  variation  of 
stresses  and  strengths  be  determined  also,  by  testing  sufficient 
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samples  to  failure.  The  result  of  such  a  test-to- failure  program 
is  illustrated  in  Figure  7-5. 

The  reader  will  note  that  on  test  number  7  the  component  is  weaker 
than  the  stress  to  which  it  will  be  subjected,  and  therefore  will 
fail . 

Obviously,  scatterbands  of  stresses  and  strengths  must  be  separated 
by  safety  margins.  Here  the  question  arises  how  large  the  safety 
margins  should  be  to  achieve  the  required  degree  of  component 
reliability. 

Before  we  may  discuss  this  vital  question,  we  must  dwell  for  the 
moment  on  the  widespread  misconception  that  reliability  may  be 
judged  on  the  basis  of  a  single  failure  test. 

Figure  7-5  indicates  that  safety  factors  fluctuate  even  more 
violently  than  the  stresses  and  strengths  upon  which  they  are 
based  (compare  tests  No.  5  and  6) .  Therefore,  relying  on  the 
test-to- failure  data  of  just  one  unit  is  shortsighted  and 
irresponsible.  This  is  illustrated  in  Figure  7-6  where  the 
scatterband  of  stress  data  has  been  replaced  by  the  maximum 
stress  level,  called  the  "Reliability  Boundary". 

If  only  one  test  were  conducted  and  relied  upon,  and  if  the 
result  complied  with  the  specified  minimum  safety  factor  of  1.5, 
as  illustrated  by  the  dot,  (T) ,  the  component  type  might  be 
accepted  for  mass  production  an<$-  employment  in  complex  military 
equipment.  If,  however,  more  units  were  tested  to  failure,  a 
shocking  degree  of  variation,  hence  unreliability,  would  be 
revealed . 

1.1  HOW  TO  JUDGE  AND  INCREASE  SAFETY  MARGINS 


The  principle  of  safety  margins  is  illustrated  by  the  examples 
shown  in  Figure  7-7. 

Let  us  assume  that  between  the  average  strength  and  the  Relia¬ 
bility  Boundary  a  minimum  safety  margin  of  five  standard  devia¬ 
tions  were  specified.  After  having  tested  a  sampl ’,  say  12  units, 
to  failure  we  compute  the  standard  deviation  and  find  that  the 
safety  margin  is  only  2.7  standard  deviations  (Figure  A).  Thus, 
the  safety  margin  must  be  increased.  We  may  first  try  to  lower 
the  severity  of  the  environmental  condition,  for  example  by 
providing  a  shock  absorber  or  by  intensifying  the  cooling  of  the 
component.  If  neither  is  practical,  the  component  must  be  re¬ 
designed.  In  most  instances,  this  is  made  easier  by  the  fact  that 
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the  failure  tests  will  have  revealed  the  prevailing  modes,  or 
mechanisms,  of  failures.  Either  the  average  strength  may  be  in¬ 
creased,  as  shown  in  Figure  B,  or  the  inherent  variation  reduced, 
as  in  Figure  C,  whichever  app  ars  most  suitable  to  save  weight, 
time,  or  expense. 

Components  having  very  large  safety  margins  may  be  considered 
"absolutely'  reliable.  They  may  be  placed  in  the  "'good1  basket", 
thereby  freeing  us  to  concentrate  on  those  component  types  which 
still  suffer  from  low  safety  margins. 

When  saving  of  weight  is  of  prime  importance,  as  in  the  design  of 
structural  components,  the  concept  of  safety  margins  permits 
saving  weight  by  keeping  the  safety  margin  down  to  the  specified 
minimum  of,  say  five  standard  deviations.  In  the  design  of  simple 
structural  parts  having  very  small  inherent  variations  of  strength 
such  as  machined  pins,  the  designer  may  reduce  dimensions  and 
weight  to  a  bare  minimum  if  he  can  prove,  tnrough  tost?  to  failure 
that  the  specified  minimum  safety  margin  of,  say  five  standard 
deviations,  is  still  available. 

Tt  thus  becomes  evident  that  the  principle  of  safety  margins  not 
only  helps  to  achieve  and  control  the  required  "absolute"  degree 
of  component  reliabilit'  ,  but  also  helps  to  improve  performance 
by  indicating  where  dead  weight  may  be  saved.  Thus  the  crucial 
antagonism  between  performance  ->nd  reliability  may  be  great  ly 
alleviated. 

1.2  HOW  MANY  STANDARD  LEV  I AT IONS? 


The  question  arises:  How  many  standard  deviations  shall  be 
specified?  Actually,  there  is  no  fixed  number  to  be  specified 
for  all  types  of  components ,  relative  to  all  env : ronmenta  1  con¬ 
ditions  and  design  cnt-ria  for  the  following  reason:  to  assure 
that  a  component  type  will  never  cause  the  loss  nt  complex 
military  equipment,  every  conceivable  risk  factors,  such  as  un¬ 
certainties  cf  measurements,  skills ,  and  of  war  conditions,  must 
be  conSi  bred.  specifying  ana  attaining  the  minimum  contingency 
margin  is  the  respons idi 1 i ty  of  the  engineer. 

Once  a  satisfactory  degree  of  design  reliability  is  establish. g, 
and  proved  to  exist  by  tests  to  failure,  the  quality  control 
engineer  will  take  over .  He  has  the  responsibility  of  assuring, 
by  approved  methods  of  statistical  quality  control,  that  during 
the  manufacturing  process  neither  the  average  strength  decreases 
nor  the  standard  deviation  increases.  He  must  prove  this  con- 
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tinuously  by  testing  to  failure  sir  ill  but  adequate  production 
samples  with  regard  to  those  environmental  conditions  which, 
during  the  prototype  tests,  have  shown  the  need  of  permanent 
control.  In  this  manner,  the  quality  control  engineer  may  main¬ 
tain,  and  even  increase,  the  safety  margins  established  in  the 
prototype  stage. 

Considering  only  the  variations  in  strength  (Figure  7-10)com- 
pared  to  a  reliability  boundary.  A  limit  may  be  determined, 
from  the  frequency  distribution,  below  which  the  strength  will 
be  found  any  given  fraction  of  trials.  (The  3  sigma  rule  when 
the  normal  distr  ib-'  tior.  applies,  is  an  example  of  this.  The 
actual  value  will  be  found  below  3  standard  deviat ions  below  the 
mean  only  .00135  of  the  time).  As  the  figure  shows,  a  contin¬ 
gency  margin  should  be  provided,  in  addition  to  the  computed 
scatter  margin  to  provide  for  unverified  assumptions.  Figure 
7-11  provides  the  complete  picture.  Stress  is  controlled  to 
keep  a  safety  margin  between  the  design  min’ mum  (probable) 
strength  and  the  design  maximum  (probable)  stress. 

1.3  OVERDESIGN  AND  RELIABILITY 

It  is  ften  argued  that  generous  safety  margins  unavoidably  lead 
to  overdesign,  that  is,  to  excessive  v.  ' i ght  ,  reduced  performance, 
high  cost,  and  delayed  schedules.  Is  this  true? 

There  is  the  per formance  fanatic  who,  by  sacrificing  reliability, 
economy  and  schedules,  tries  to  squee-e  out  of  his  design  the 
ultimate  degree  of  performance,  the  maximum  output.  There  is 
the  onr esour ce fu 1 .  apprehensive  designer  who  clings  to  his 
design,  unable  to  finish  and  release  it  for  production.  In 
either  case,  warnings  agai  ist  overdesign  are  well  justified. 

But  there  is  also  the  hasty,  superficial  desi  iner  who,  pretending 
to  fight  against  overdesign,  tries  to  push  a  new  design  into 
production,  be  it  mature  or  immature,  light  or  heavy  ,  inexpensive 
or  expensive,  reliable  or  unreliable. 

Significantly,  advocates  of  haste  and  superficiality  are  the 
ones  who  assert  that  reliability  may  be  improved  later,  during 
production  and  service  use,  by  quality  control  and  failure  re¬ 
porting.  .Since  this  is  impossible,  they  just  bring  about  the 
very  consequences  of  overdesign  they  pretend  to  battle,  namely 
excessive  weight,  reduced  pier  formance ,  high  cost  and  --  as  ■* 
result  of  necessary  design  changes  --  badly  delayed  schedules. 
Worst  of  all,  they  bring  about  poor  reliability. 
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While  warnings  against  overdesign  are  oftentimes  justified,  they 
must  never  be  misconstrued  as  an  invitation  to  neglect  the  prin¬ 
ciple  of  safety  margins.  Whenever  this  is  the  case,  the  engineer 
must  take  immediate  action,  education  or  otherwise,  before  a  low 
reliability  barrier  becomes  chronic  and  incurable. 


2.  STRESS  AND  STRENGTH  DISTRIBUTIONS 

For  the  purpose  of  discussing  the  concepts  of  stress-strength 
analysis,  we  have  used  a  normal  distribution  in  our  examples. 

The  assumption  that  stresses  and  strengths  are  normally  distri¬ 
buted  is  not  necessarily  valid.  In  using  the  stress-strength 
approach,  this  assumption  is  dangerous  (much  more  than  in  estim¬ 
ating  mean  time  between  failures,  for  example)  because  the  com¬ 
parison  is  being  made  well  out  on  the  tail,  in  the  extreme  value 
region.  Other  possible  distributions,  approximating  the  normal, 
such  as  Poisson,  Gamma,  Weibull,  or  distributions  like  the  log¬ 
normal,  are  also  eligible  candidates.  The  identification  and 
testing  of  distributions  is  covered  in  chapter  10.  We  will 
discxiss  a  generalized  distribution  here  to  guide  the  use  of 
probability  theory  to  the  establishment  of  safety  margins. 

2.1  JO INT  D ISTRIBUT  IONS 


If  you  assume  a  large  number  of  tests  of  the  strength  of  a  given 
manufactured  part,  each  test  being  run  to  failure,  some  relation¬ 
ship  between  the  number  failing  at  any  particular  value  of 
strength  (or  band  of  values)  and  the  value  can  be  determined. 

This  is  called  a  frequency  distribution  or  density  function 
(Figure  7-13).  If  the  exact  relationship  were  known,  you  could 
predict  the  probability  of  a  randomly  selected  specimen  failing 
at  a  particular  value  of  stress  f'.  It  would  be  that  fraction 
of  the  population,  whose  strength  was  equal  to  or.  less  than  a 
stress  f'.  Similarly,  if  you  conducted  an  experiment  a  large 
number  of  times,  recording  the  stress  on  each  experiment,  a 
relationship  between  the  relative  frequency  (or  density)  of 
stresses  and  the  stress  could  be  established.  If  the  exact  re¬ 
lationship  were  known,  you  could  predict  the  probability  that 
on  any  randomly  selected  trial  (Figure  7-14)  the  stress  would 
exceed  a  strength  s'.  This  would  be  the  fraction  of  the  popula¬ 
tion  (of  possible  trials)  in  which  the  stress  exceeded  the 
strength  S'.  These  fractions  are,  of  course,  the  ratio  of  the 
areas  under  the  curve  to  the  left  of  F'  or  right  of  s'  to  the 
total  area  under  each  respective  curve.  If  the  two  curves  are 
"normalized" ,  that  is  if  the  ordinates  on  the  curve  are  divided 
by  a  common  factor  such  that  the  total  area  under  the  curve  is 
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Look i nq  at  Figure  7-lb,  the  probability  that  the  strength  will 
be  S  on  a  particular  part  is  the  area  under  the  curve  F(S)dx. 

The  probability  that  the  stress.  F.  is  equal  to  or  greater  than 
the  strength,  c,  om  any  particular  experiment  is  the  area  under 
the  tai 1 

•t> 

!  F(F)dF 
s 

The  probability  that  a  failure  will  occur  is  the  probability 
that  S  -  x  and  F  >  x.  This  is  the  product  of  the  individual 
probabilities.  So  the  probability  that  a  failure  will  occur  (Q) 
is 

OD 

j*  CD 

Q  -  J  F  ( S )  r  r  F  (  F)  dF "]  dx 
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This  equation  can  be  solved  analy ciectily,  graphically,  by  numer¬ 
ical  integration  or  by  probabi list ic  techniques  such  as  "Monte 
Carlo"  provided  the  form  or  shape  of  the  probability  distribu¬ 
tion  functions  F{S)  and  F(F)  can  be  determined  (chapter  10). 

2  :2_ _ STRESS -ST RKNGT  ~T  ANALYS  IS  FOR  THE  NORMAL  DISTRIBUTION 

2.2.’  Analytical  Baslo;  If  both  distributions  are  Normal 
(gauss ian)  an  «u«iyt  cal  solution  has  been  ucvelopeJ.  netting 
S  be  the  mean  -alue  f  the  strength  with  standard  deviation  s 

f  its  standard  deviation 
and  F  are 


F(F)  - - e 

•  2  ”  f 

If  we  designate  D  5  -  F  then  the  reliability  (the  probability 
that  s  >  p)  can  be  determined  from  the  equation  D  ~  S  -  F  >0. 
f(d)  is  defined  as  the  difference  distribution  of  F(S)  and  F(F). 
F  (D)  is  also  normally  distributed  (3). 
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The  reliability  is  given  by 
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2.2.2  Appl  icat  ion :  The  method  basically  involves  the  ste^>» 
outlined  in  Figure  7- IB  and  discussed  herewith: 


A.  Determine  Approximate  Design:  Since  the  method  involves 
prediction  of  reliability  from  geometry  of  the  design,  a  tenta¬ 
tive  configuration  must  be  established.  As  the  analysis  pro¬ 
gresses,  the  design  is  corrected  and  refined  to  satisfy  the 
or iteria. 


B.  Determine  Critical Stresses .  Since  all  stresses  in  a  design 
do  not  lead  to  failure,  v.e  must  fiiSt  select  and  quantify  those 
stresses  that  will  cause  failure  if  vhey  exceed  achievable 
strength.  The  word  "critical"  is  used  to  denote  these.  The 
following  steps  are  involved: 

1.  Determine  the  nominal  stresses,  each  as  a  function  of 

loads  (normal  and  shear),  temperature,  geometry,  physic*! 
properties  (Poisson's  ratio.  Young's  modulus,  shear 
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modulus,  thermal  expansion,  thermal  conductivity)  and 
time  (stress  oycleo  vs.  life). 

2.  Determine  factors  affecting  maximum  stress  such  as  ( 5 >  a 
('*)  stress  concentration  factors  (b)  load  factors  such 
as  static,  dynamic,  impact,  shock,  and  energy,  (c)  tem¬ 
perature  stress  factors  around  critical  points,  (d)  manu¬ 
facturing  stress  factors  such  as  for  machining,  grinding, 
extruding,  and  drawing,  (e)  surface  treatment  stress 
factors  such  as  for  shot  peening,  cold  working,  and  plat¬ 
ing,  (f)  heat  treatment  stress  factors  via  distortion, 

(g)  assembly  stress  factors  such  as  for  shrink  and  press 
fits,  (h)  notch  sensitivity  factors,  particularly  in 
fatigue,  (i)  environmental  stress  factors  such  as  surface 
corrosion  and  gross  temperature  effects.  When  these  are 
appropriately  combined  with  the  basic  nominal  stress,  the 
effect,  as  shown  in  Figure  7-20,  is  to  establish  a  higher 
critical  mean  stress. 

3.  Calculate  all  critical  stress  components  :  First  determine 
which  of  the  stresses,  considering  the  above  factors,  are 
likely  to  be  critical  (i.e.  approach  strength  and  cause 
failure  if  they  do).  Then  for  each  calculate  all  three 
normal  and  all  three  sheaf  stresses.  While  the  appropriate 
stress  factors  are  applied. 

4.  Calculate  critical  mean  stresses,  such  as  maximum  tensile 
stress,  shear  stress,  or  distortion  energy,  or  the  com¬ 
bination  of  mean  and  alternating  fatigue  stresses. 

5.  Determine  critical  stress  distributions  for  useful  life¬ 
time.  This  can  be  done  by  listing  all  the  principal 
application  situations,  the  environment  for  each,  the  re¬ 
sultant  critical  stress  for  each,  and  the  estimated  per 
cent  of  lifetime  that  it  will  encounter  each  situation. 
Then  a  normal  (or  other)  density  function  can  be  fitted 
to  the  data  by  regression,  and  (if  normal)  the  standard 
deviation  obtained. 

C.  Determine  Material  and  its  Unit  Strength.  Here  much  depends 
upon  the  criterion  for  strength  beyond  which  failure  is  defined 
to  have  occurred: 

1.  Determine  all  critical  unit  strength  mean  values:  Select 
one  or  more  suitable  materials.  Then  determine  (a)  direct 
stresiVa train  criteria  (ultimate  strength,  yield  strength, 
or  proportional  limit,  depending  upon  application) r 
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(b)  shear  strength  in  the-  same  way,  (c)  distortion  energy 
strength,  and  (d)  fatigue  strength. 

2.  Determine  factors  that  affect  strength ,  such  as  (a)  size 

and  load,  (b)  manufacturing  processes,  (c)  heat  treatment, 
(d)  surface  treatment,  (e)  environn.ent  (temperature, 
humidity,  corrosion,  etc.),  and  (f)  time  effects  (aging, 
cold  flow,  fatigue,  and  corrosion:  Figure  7-19  shows 

the  general  strength  reduction  due  to  these  factors. 

3 .  Determine  actual  unit  strength  means  and  distributions. 
Apply  the  appropriate  strength  factors  to  determine  the 
net  mean  strength  for  the  application  conditions.  Then 
determine  the  distribution  for  each  from  the  -material 
suppliers  or  testing  laboratories,  or  conduct  tests-to- 
failure  as  necessary.  Again  fit  a  normal  (or  other) 
density  function  to  the  data  by  regression,  and  (if 
normal)  obtain  the  standard  deviation. 

D.  Determine  the  Required  Strengths;  Now  that  we  have  the 
anticipated  stresses  and  the  material  unit  strengths,  we  can 
proceed  to  determine  the  total  strengths  required  for  adequate 
reliability.  But  first  let's  examine  the  stress/strength  re¬ 
lationship. 

If  we  were  to  conduct  a  series  of  25  tests  of  a  critical  stress 
within  a  given  design,  they  might,  fall  in  a  "scatterband"  as 
shown  in  Figure  7-22.  if  these  stress  points  are  "normally" 
distributed,  68%  will  fall  within  a  band  of  say  t  2  kips  each 
side  of  the  example  mean  13  kips,  another  27%  will  fall  within 
the  next  2-kip  bands  on  each  side,  another  4%  in  the  next  2-kip 
bands,  etc.  This  is  expressed  by  the  area  under  the  standard 
density  function  curve  at  the  right.  The  "standard  deviation" 
of  this  normal  distribution  is  f  -2  kips. 

Now  if  we  were  to  conduct  a  similar  series  of  v_ests-to- failure 
to  get  strength  of  the  material,  we  typically  would  find  the 
same  shape  of  curve,  but  some  other  value  of  standard  deviation. 
For  the  example  it  is  s  =  0.5  kip.  And  we  now  see  that  the  mean 
values  are  separated  by  D  =  5  kips. 

Now  the  overlap  of  the  two  curves  tells  us  that  if  we  were  to 
conduct  enough  tests,  cr  encounte.  enough  operational  situations, 
sooner  or  later  we  will  get  a  stress  point  exceeding  strength, 
and  we  should  have  a  failure.  The  probability  that  this  will 
not  occur,  for  normal  distribution:,,  as  we  have  seen 
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Reliability  R  =  - 
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which  is  the  area  under  the  normal  density  function,  available 
in  many  books  (7)  to 


Extensions  to 

/  D  \  A 

(d>  =  u-4 

can  be  obtained  from  references  (3)  using 


x  -  (§)A/  2  and  R  =  1  -  ( 1-area) /2 , 
a 

Since  R  provides  an  unwieldy  string  of  9s,  it  is  usually  more 
convenient  to  express  unreliability  U  =  1  -  R.  D  is  the  differ¬ 
ence  between  mean  strength  and  stress,  and  s  and  f  are  the  re¬ 
spective  standard  deviations.  Figure  7-23  gives  the  resultant 

relation  of  U  to  (^-)  .  With  this  background  we  can  outline  the 
procedure: 

1  •  Trans  1 ate  reliability  requirement  to  Reliability  Margin % 
Calculate  U  from  the  specified  reliability  R  =  1  -  U.  Use 

,D. 

Figure  7-21  to  find  the  required  Reliability  Margin  (— ) . 

2.  Calculate  mean  stress/s trengtn  variance:  Use  the  standard 
deviations  s  and  f  obtained  from  R5  and  C2  above,  to  obtain 
their  mean  d  -  J  s2  +  fs. 

,D.  . 

3.  Calculate  the  required  mean  strengths,  by  adding  D  -  {—} ’d 
to  mean  stress.  Now  we  know  what  strength  is  needed 

to  achieve  the  required  reliability. 

E.  Determine  Size  and  Shape:  Now  that  the  material  and  its  unit 
strengths  have  been  established,  and  the  required  strengths  cal¬ 
culated,  we  can  proceed  to  design  for  adequate  size  and  shape  to 


} 

4 


7-25 


achieve  the  required  strengths: 

1.  Select  or  design  fur  the  section  modulus  required,  using 
standard  section  handbooks  and  established  design  calcu- 
lat ions . 

2.  Modify  the  design  and/or  the  materials  until  all  Relia¬ 
bility  Margins  are  met. 

F.  Verify  the  Design  Reliability  Margin:  Nearly  all  design 
involves  many  assumptions  to  avo'd  unjustifiable  volume  of 
analysis  or  test  cost.  The  above  approach  permits  design  to 
predictable  reliability,  but  does  not  insure  against  design 
err'irs  of  assumption,  analysis,  omission,  etc.  Verification 
is  covered  in  chapter  13. 

2.2.3  Other  Distribution:  The  preceding  approach  assumed  that 
both  stress  and  strength  distributions  were  normally  distributed, 
that  is,  could  be  described  by  the  normal  (gaussian)  distribution. 
;,s  mentioned  earlier,  this  is  a  very  dangerous  assumption.  Where 
a  distribution  can  be  established,  analytical  solutions  can  be 
derived.  Reference  (5)  provides  a  very  useful  listing  of 
references  for  special  distributions. 

The  computation  of  the  difference  functions  and  the  determina¬ 
tions  of  reliability  have  been  analytically  established  for  the 
log  normal  (4),  Gamma  (5)  and  Weibull  (6)  distributions.  Refer¬ 
ence  5  also  suggests  alternatives  such  as  conformal  mapping  or 
numerical  or  graphical  integration  for  obtaining  solutions  in 
special  cases. 

2.2.4  Stress/strength  testing :  When  distribution  data  is  not 
obtainable  for  the  above  analytical  approach,  yet  the  design 
reliability  is  a  critical  matter,  it  may  be  necessary  to  conduct 
experimental  tests.  Tests  to  determine  stress  distribution  in  a 
prototype  are  fairly  straightforward  and  non- destructive ,  using 
instrumentation  such  as  strain  ga^es,  plastic  models  and  polar- 
ized  light,  etc.  To  the  extent  that  such  tests  can  simulate  the 
manufacturing  variances,  operational  environment,  external 
stresses,  and  time  effects,  the  results  can  be  quite  dependable. 

But  tests  of  strength  distribution  are  much  more  difficult,  ex¬ 
pensive,  and  time  consuming.  If  the  design  engineer  can  identify 
specific  local  areas  of  critical  doubt,  a  series  of  comparatively 
simple  tests  can  be  designed,  wherein  stress  is  repeatedly  in¬ 
creased  until  failure  occurs,  providing  a  rough  strength  distri¬ 
bution  curve  for  the  local  are-*.  On  the  other  hand  it  may  be 
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more  convincing,  if  not  morn  economical  to  test  an  entire  proto¬ 
type  in  the  same  manner,  so  that  all  interactions  are  accounted 
for.  repairing  failures  each  time  they  occur.  Of  course  as 
strength  inadequacies  are  thus  brought  tc  light,  the'  design  is 
changed  to  get  required  str  ngth. 

Such  stress/strength  testing  should  not  be  confused  with  simple 
"overstress"  testing,  which  determines  only  that  the  design  dees 
not  fail  at  some  specified  stress  above  the  operational  level. 
Over-stress  testing  does  not  generally  determine  strength. 

3.  APPLICAT  IONS 

The  use  of  the  "safety  margin"  approach  is  an  improvement  over 
the  "safety  factors"  approach  in  that  it  provides  an  analytical 
method  of  evaluating  the  risk  that  an  overstress  or  understrength 
condition  will  exist.  Instead  of  a  pyramid  of  safety  factors 
imposed  by  each  area  providing  the  "worst  case"  value,  the  pro¬ 
bability  of  failure  is  evaluated  on  the  distribution  of  values. 
Figure  7-26  illustrates  the  comparison.  The  strength  of  material 
is  quoted  at  the  -3c  value,  the  computation  of  stresses  is  made 
at  the  moan  value,  the  safety  factor  used  is  5.  By  evaluation 
of  the  distributions  *-he  safety  factor  could  very  realistically 
have  been  set  at  3. 

The  analysis  attempts  to  evaluate  the  probability  of  finding  a 
value  of  stress  much  larger  than  (or  strength  much  less  than)  the 
nominal  value.  Where  this  probability  is  high,  the  safety  margin 
must,  be  great,  where  the  probability  is  low,  a  small  margin  will 
suffice.  Where  this  probability  is  not  capable  of  estimation, 
approximations  must  be  used  and  a  contingency  factor  based  on  the 
objective  knowledge  obtained  from  testing  or  analysis  applira. 

The  purposes  of  safety  margin  analysis  is  to  improve  the  com¬ 
petitive  position  of  the  design;  that  is,  to  find  the  optimum 
comparison  of  stress  and  strength  that  will  (a)  have  an  accep¬ 
table  probability  of  success  and  (b)  comp  e  favorably  with 
other  C'ses*  i  lints  such  as  weight,  cost,  availability  of  material. 
My  fave'-ite  example  is  the  assumption  made  years  -’go  that  brick 
and  mortar  c'ulg  stand  a  tensile  load.  Tests  confirmed  this  and 
using  a  tenet le  stress  loading  in  the  design  of  large  furnace 
chimneys  of  one  pound  per  square  foot,  the  industry  was  revolu¬ 
tionised  by  the  appearance  of  tall,  skinny  (to  them)  chimneys. 


Recognizing  that  while  in  most  shipbuilding  material  the  standard 
deviations  of  strength,  with  the  usual  manufacturing  ccr.  rc  1  and 
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inspection,  is  negligible-  in  comparison  to  the  yield  and  ultimate 
strengths,  the  stresses  imposed  by  dynamic  loading  may  be  highly 
probabilistic  in  nature.  There  were  a  number  of  Jeep  aircraft 
carrier  that  suffered  damage  to  their  flight  decks  during  the 
war  due  to  heading  into  seas  during  hurricane  force  storms. 

Cons ideration  was  given  to  greatly  strengthening  the  structural 
support.  A  decision  was  made  (in  the  CNO  as  recommended  by  the 
Bureau)  that,  the  probability  that  the  situation  would  need  to 
occur,  that  carriers  would  need  to  recover  aircraft  in  a 
hurricane,  was  small  enough  to  make  the  (then)  present  design 
acceptable.  Fleet  and  Task  Group  commanders  were  informed  of 
the  limitation  on  the  ships  cupabi lity  and  told  to  avoid  the 
situation. 

The  selection  of  the  appropriate  working  stress,  for  com¬ 
petitive  design,  snouid  consider  the  nominal  maximum  loading 
anticipated  but  should  also  consider  the  distribution  of  loadings 
which  may  cause  stresses  in  excess  of  this  value  (as  probability 
that  a  ship  must  proceed  on  a  particular  course  with  relation  to 
a  hurricane).  The  strength  computation  should  be  based  on  an 
acceptable  value  of  risk,  as  opposed  to  a  nominal  stress  value 
hoping  the  safety  factor  is  adequate  to  prevent  failure  when 
the  extreme  occurs. 

Use  of  the  stress -strength  approac..  provides  the  engineer  with 
one  more  analytical  tool  to  assist  in  reaching  decisions  in  the 
process  of  design  and  development  of  systems. 
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Chapter  8 
MAINTAINABILITY 

Maintainability  concepts  are  being  emphasized  in  all  services  of 
the  Department  of  Defense  because  of  the  high  costs  associated 
with  maintaining  equipment  operational.  Three  closely  assoc¬ 
iated  problems  have  increased  maintenance  costs  within  the  Navy. 
Following  World  War  II  came  a  tremendous  increase  in  complexity 
of  ships  equipment.  And  ships  became  pero  specialized.  This  was 
accompanied  by  an  increase  in  the  turnover  of  personnel.  As  the 
equipment  became  more  difficult  to  maintain,  the  capabilities  of 
the  maintenance  personnel  fell  behind;  they  were  less  able  to 
cope  w  i  i;  h  the  problems.  The  approach  being  taken  by  the  services 
is  to  increase  the  maintaina*..  '  litv  of  systems. 

Maintainability  is  the  speed  or  economy  with  which  ?  system  or 
component  can  be  kept  in,  and/or  restored  to,  full  performance 
capability.  A  principally-used  measure  is  the  average  number  of 
failures  restored  per  hour  of  Corrective  Maintenance  time,  which 
is  the  reciprocal  of  MTTR .  mot!  tr  is  the  fraction,  of  attempts 
wherein  restoration  is  completed  in  a  specified  time,  or  the  pro¬ 
bability  that  it  will  be  completed  in  that  time.  Another  is  the 
operational  time  per  dollar  cost  ot  preventive  and  corrective 
maintenance . 

The  objectives  of  a  maintainability  program  include  the  perfection 
of  the  design  to  assure  that  maintenance  actions  can  be  accomplish¬ 
ed  in  minimum  time,  with  minimum  effort  but  with  maximum  safety. 

By  the  above  definition,  we  find  ourselves  concerned  with  four 
areas  of  enquiry: 

(a)  The  capabilities  and  characteristics  (both  mental  and 
physical)  of  the  people  who  maintain  and  operate  the 
system , 

(b)  The  design  of  equipment  suited  to  the  characteristics  of 
such  people, 

(c)  The  quantification  of  requirements,  prediction,  and 
vc  Miration  to  control  the  achievement  of  maintain¬ 
ability  and  assure  ourselves  that  the  system  meets  our 
needs ,  and 

(d)  The  management  control  of  maintenance  resources. 
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Maintainability  is  often  confused  with  maintenance.  The-  achieve¬ 
ment  of  maintainability  is  a  design  function,  but  maintenance  is 
a  consequence  of  design  and  use.  There  are  two  other  similar 

terms  that  can  be  confusing;  maintainability  analysis  and  main¬ 
tenance  analysis.  The  design  function  that  analyzes  equipment 
and  systems  to  determine  what  operation  and  maintenance  actions 
are  required  to  keep  equipment  or  systems  operating  does  main¬ 
tainability  analysis.  Analysis  of  maintenance  tasks  to  determine 
the  resources  required  to  do  the  work  is  maintenance  analysis. 
Resources  again  mean  men,  money,  material,  facilities,  time  and 
morale . 

One  military  specification  (5)  has  the  following  requirements  for 
maintainability  analysis  and  maintenance  analysis. 

"Maintainability  Analysis,  A  maintainability  engineering 
analysis  of  the  system  shall  be  accomplished  concurrently 
with  the  design  effort.  This  analysis  shall  provide  a 
definition  of  maintainability  design  features  to  be  in¬ 
corporated  in  the  hardware.  This  analysis  shall  oe  used 
to  evaluate  the  degree  of  achievement  of  the  maintainability 
design  goals,  including  inherent  mean  and  maximum  down  time, 
the  logistic  and  personnel  subsystems  decisions  related  to 
support  cost  of  the  system  versus  design  and  support  alter¬ 
natives.  The  primary  inputs  into  the  maintainability  analy¬ 
sis  will  be  data  obtained  from  design  engineering  reports, 
data  and  studies  prepared  by  the  contractor,  and  the  require¬ 
ments  furnished  bv  the  procuring  agency." 

"Maintenance  Analysis.  The  contractor  shall  conduct  a  detail¬ 
ed  determination  of  hardware  maintenance  tasks,  tools  and  test 
equipment,  and  spares  line  item  identification.  This  is  a 
portion  of  the  ove^-all  system  analysis  and  provides  feedback 
to  the  maintainability  analysis." 

It  is  obvious  that  the  two  types  of  analysis  cover  the  same 
ground.  The  designer  must,  in  his  design  foresee  the  maintenance 
tasks  that  will  be  required  to  maintain  and  operate  the  equipment 
if  he  is  to  incorporate  features  into  the  design  to  improve  the 
ease  and  economy  of  repair  or  maintenance.  For  optimization  of 
the  maintainability  of  the  design  includes  design  for  minimum 
support  requirements  as  well  as  access  and  simplicity  of  required 
operations.  To  achieve  this  will  require  a  rather  detailed  task 
analysis  which  will  be  partially  duplicated  by  the  subsequent 
maintenance  analysis.  Reasonable  efficiency  requires  that  the 
twe  c  forts  be  married  from  the  concept  of  the  design  not  only 
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to  reduce  duplication  of  effort,  but  to  prevent  different  main¬ 
tenance  concepts  from  being  developed,  as  will  usually  occur 
unless  the  two  are  coordinated. 

Accepting  that  the  perfect  machine  -  one  designed  to  perform  its 
function  whenever  called  upon  and  never  to  have  a  failure  -  has 
yet  to  be  designed,  we  realize  we  must  accept  something  not  quite 
perfect.  But  how  much  less  than  perfect?  The  answer  must  be 
based  or.  the  function  the  system  is  required  to  perform. 

There  are  two  primary  roads  we  can  follow.  We  can  spend  every 
dollar  we  can  afford  to  make  the  system  reliable  -  to  reduce  the 
the  incidence  of  failures  so  that  it  almost  never  needs  to  be 
repaired.  Or  we  can  permit  the  system  to  fail,  as  often  as  it 
needs  to,  spending  our  money  in  the  design  to  make  it  almost 
instantly  restorable.  This  second  approach  is  called  maintain¬ 
ability.  As  might  be  expected,  the  best  and  most  economical 
approach  is  usually  somewhere  between  these  extremes. 

Lets  take  an  example.  The  functional  requirements  in  terms  of 
consistency  of  performance  are  different  for  the  refrigerators 
on  a  freighter  than  for  the  steering  engines  of  the  same  ship. 

The  consequences  of  failure  in  the  steering  engines  are  immediate 
-  lack  of  control,  usually  with  the  rudder  hard  over.  Collision 
or  grounding  are  predictable  consequences.  The  refrigerator,  on 
the  other  hand,  can  maintain  a  low  temperature  for  a  considerable 
time.  Ultimately  spoilage  and  logistic  problems  may  result  if  not 
repaired  soon  enough ,  but  immediate  consequences  are  not  foreseen. 

In  the  case  of  the  steering  engine,  we  demand  high  reliability, 
a  very  low  failure  rate.  To  achieve  this  we  provide  duplicate 
systems  so  that,  should  one  fail,  the  other  can  be  used.  For  the 
refrigerator,  no  such  instantaneous  replacement  is  required.  We 
rather  require  that  the  equipment  be  operable  a  high  percentage 
of  the  time,  v'ith  no  extemely  long  down  times.  This  latter 
characteristic  we  define  as  maintainability.. 

1 .  AVAILABILITY 

1.1  INHERENT  AVAILABILITY 

The  point  of  comparison  between  reliability  and  maintainability 
as  a  design  approach  is  called  Availability.  Availability  is 
the  action  of  the  total  desired  operating  time  that  the  system 
component  is  operable  (chapter  27).  For  prediction 
purposes  it  is  also  the  probability  that  a  system  or  equipment 
is  operating  satisfactorily  at  any  point  in  time  when  used  under 
stated  conditions  (1).  We  might  consider  a  system  such  as  the 
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evaporators  on  board  ship.  The  requirement  for  operation  depends 
on  the  storage  capacity  and  usage  of  fresh  water.  At  any  point 
in  time  the  evaporators  may  be  operating.  If  they  are  not, 
several  possible  reasons  may  account  for  their  shutdown: 

(a)  Water  tanks  are  full; 

(b)  Inadequate  auxiliary  exhaust  steam  makes  their  use  un¬ 
economical  ; 

(c)  Polluted  harbor  water  makes  operation  undesirable; 

(d)  Evaporators  are  down  for  maintenance; 

(e)  Evaporators  are  down  for  repair. 

Considering  only  the  last  reason.  Figure  0-6  provides  a  pictorial 
explanation  of  Inherent  Availability,  A^ . 

Inherent  Availability  (4)  is  the  fraction  of  total  time  that  a 
system  or  equipment,  when  used  under  stated  conditions  in  an  ideal 
supply  environment,  is  capable  of  operation.  Inherent  Avail¬ 
ability  excludes  time  down  except  for  the  time  necessary  tc  diag¬ 
nose  the  trouble,  repair  the  fault,  test  out  and  restart  the 
equipment . 

The  two  components  of  Inherent  Availability  are  Reliability  and 
Maintainability.  Reliability  can  be  measured  in  terms  of  Mean 
Time  Between  Failures  (MTBF) .  Maintainability  is  measured  in 
terms  of  Mean  Time  to  Restore  (MTTR) .  Restoration  is  used  in 
preference  to  repair  since  restoration  ' ?  used  in  the  sense  cf 
returning  the  system  to  operation  by  using  replacements  or  possi¬ 
bly  by  switching  on  redundant  elements,  where  repair  may  include 
welding  a  crack,  or  depot  or  factory  repair  of  replaced  modules 
subsequent  to  their  removal.  The  measure,  MTTR,  is  defined  as 
the  statistical  mean  of  the  distribution  of  times  to  restore.  The 
summation  of  active  restoration  times  during  a  given  period  of  time 
divided  by  the  total  number  of  failures  during  the  same  time  interval. 

On  the  average,  the  equipment  will  operate  a  time  equal  to  the 
MTBF  before  failure.  On  the  average  the  equipment  will  be  restored 
to  operating  condition  in  a  time  equal  to  the  MTTR.  The  average 
time  during  which  the  equipment  may  be  considered  available  is 
the  fraction  of  the  total  time  represented  by  the  equations 

_  MTBF 
Ai  MTBF  +  MTTR 
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1.2  0 PER AT IONAL  AVAILABILITY 

As  pointed  out  earlier,  it  may  not  be  possible  to  operate  the 
equipment  this  fraction  of  the  time.  Most  equipments  require 
some  down  time  for  routine  (scheduled)  maintenance.  Lack  of 
spare  parts  or  lack  of  manpower  may  delay  the  restoration  action. 
Or  administrative  reasons  may  require  the  equipment  to  be  shut 
down.  The  following  terms  are  applied  to  non-operating  time  (4)  : 

(a)  Downtime :  That  portion  of  calendar  time  during 
which  the  item  is  not  in  condition  to  perform 
its  intended  function. 


(b)  Preventive  Maintenance  Time:  The  maintenance  time  to 
retain  an  item  in  satisfactory  operational  condition  by 
providing  systematic  inspection,  detection,  and  preven¬ 
tion  of  incipient  failure.  It  is  made  up  of  perform¬ 
ance  measurement,  care  of  mechanical  wearout  items, 
front  panel  adjustment,  calibration  and  alignment, 
cleaning,  etc. 

(c)  Corrective  Maintenance  Time:  The  time  that  begins  with 
the  observance  of  a  malfunction  of  an  item  and  ends  when 
the  item  is  restored  to  a  satisfactory  operating  con¬ 
dition.  It  may  be  subdivided  into  Active  Maintenance 
Time  and  Non-Active  Maintenance  Time. 

(d)  Active  Restoration  Time:  The  Corrective  Maintenance 
Time  during  which  work  is  actually  being  done.  It  in¬ 
cludes  detection,  diagnosis,  preparation,  replacement 
or  repair,  adjustment,  checkout,  and  reload  time  to  the 
extent  each  is  necessary. 

(e)  Active  Maintenance  Time:  The  time  during  which  preven¬ 
tive  and  corrective  maintenance  work  is  actually  being  j 

done  on  the  item.  ' 

(f)  Non-Active  Maintenance  Downtime:  The  time  during  which  ■ 

no  maintenance  is  being  accomplished  on  the  item 

because  of  either  supply  or  administrative  reasons. 
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(g)  Administrative  Time:  That  portion  of  Non-active  Main¬ 
tenance  Time  that  is  not  included  in  Supply  Time. 

(h)  Supply  Time:  That  portion  of  Non-Active  Maintenance 
Time  during  which  maintenance  is  delayed  solely  because 
a  needed  item  is  not  immediately  available. 

(i)  Inactive  Time:  The  period  of  time  when  the  item  is 
available,  but  is  neither  needed  nor  operating  for  its 
intended  use. 


The  term,  Operational  Availability  (A0),  is  used  to  describe  that 
fraction  of  the  total  time  that  the  system,  when  used  under  stated 
conditions  in  an  actual  supply  environment,  will  operate  satis¬ 
factorily  when  required.  Supply  time  and  Administrative  time 
are  included. 


2. 


REQUIREMENTS 


2.1  BASIC  APPROACHES 

OPNAVINST  3910. 4A  outlines  how  the  Navy  prepares  Technical  Devel¬ 
opment  Flan  Summaries.  Enclosure  1  to  the  instruction  defines  the 
information  to  be  included  in  sections  of  the  summary.  Sections 
10  through  13  require  information  pertaining  to  maintainability. 
The  four  sections  are  titled: 


Section  10 
Section  11 
Section  12 
Section  13 


Dependability  Plan 

Operability  and  Supportabi lity  Plan 
Test  and  Evaluation  Plan 
Personnel  and  Training 


The  Dependability  Plan  of  Section  10  sets  up  Availability  and 
Operational  Readiness  Goals.  Quantitative  Reliability  and  Main¬ 
tainability  goals  (MTBF  and  MTTR )  can  be  set  up  from  the  avail¬ 
ability  and  readiness  goala.  At  the  start  of  a  project  or  in  the 
project  Definition  Phase,  only  gross  statements  can  be  made  for 
maintainability.  Like  system  analysis,  maintainability  analysis 
is  an  iterative  process  which  gets  progressively  refined  is  a 
project  progresses. 


i 


8-9 


» 


» 


A<_  the  end  of  a  Project  Definition  Phase  the  following  should  be 
established  for  Maintainability  Assurance: 

(a)  A  maintenance  philos '"~>hy  is  described  for  the  system  to 
provide  essential  data  for  the  Suppor t ability  I  Ian  and  the  Person¬ 
nel  Training  Plan-  The  maintenance  philosophy  will  develop: 

(1)  Echelons  or  levels  of  maintenance,  including  maintenance 
tasks  and  skills  for  each  level. 

(2)  Planned  use  of  built-in  maintenance  aids  such  as  self¬ 
test  features,  malfunction  indicators,  specialized  or  standard 
test  equipment,  etc. 

(3)  Planned  use  of  job  aids  such  as  troubleshooting  logic 
charts,  system  technical  manuals,  audio-visual  presentation 
of  maintenance  tasks,  etc. 

(4)  Other  design  features  which  may  affect  spare  parts  and 
repairs  such  as  use  of  standard  circuits  from  specific  hand¬ 
books,  disposable  modules,  etc. 

(5)  Unique  knowledge  of  skills  required  by  the  system. 

(6)  Equipment  utilization  or  operational  cycle. 

(7)  Maintenance  environment. 

(8'  Maintenance  facilities. 

(b)  Applicable  MIL  specifications  are  defined. 

(c)  Uuunti f ication  of  Maintainability,  i.e.,  development  and 
application  of  numerical  measures  of  maintainability. 

(1)  Mean  Time  to  Restore  (MTTR) 

(2)  Maximum  Time  to  Restore  (MAXTR) 

(3)  Other 

(d)  Maintainability  apportionment  and  prediction.  This  involves 
the  allocation  of  over-all  system  measures  of  maintainability  to 
all  major  lower-order  elements  of  the  system,  with  special  regard 
for  maintenance  tasks,  times  and  test  equipment  required  at  the 
various  echelons  involved.  It  also  includes  data  concerning  the 
extent,  schedule,  design,  influence,  etc.  of  prediction  in  the 
over-all  plan  for  Maintainability  assurance. 
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(e)  Maintenance  tasks  and  skill  analysis. 

(f)  Maintainability  desiqn  reviews. 

(g)  Test  and  demonstration. 

(h)  Maintenance  data  collection,  feedback  and  analysis. 

The  maintainability  assurance  plan  will  vary  in  complexity  with 
the  size  of  the  project  and  careful  evaluation  has  tc  be  made  cf 
the  oenetits  to  be  received  from  the  expenditures  to  be  made. 

A  proposed  DOD  instruction  titled  "Development  of  the  Weapon 
System  or  Equipment  Integrate^  Support  Package,"  defines  the  role 
of  the  material  manager  in  and  the  minimum  requirements  for  the 
systematic  and  orderly  development  of  the  weapon  system  or  equip¬ 
ment  integrated  support  package." 

The  elements  of  an  Integrated  Support  Package  are: 

1.  Planned  Maintenance 

2.  Logistics  Personnel  Subsystem 

3.  Logistics  Data 

4.  Support  Equipment 

5.  Spares  and  Repair  Parts 

6.  Facilities 

7.  Contractor  Support 

It  can  be  seer,  that  the  elements  of  an  Integrated  Support  Package 
are  closely  aligned  to  the  Maintainability  Assurance  portion  of 
the  Dependability  Plan  outlined  in  OPNAVINST  3910. 4a. 

2.2  SPECIFICATIONS 

All  of  the  services  are  implementing  the  DOD  directive  and  ins¬ 
tructions  with  speci f ic3t ions  and  handbooks  on  maintainability. 
MIL  M  23313  (SHIPSW2)  outlines  a  comprehensive  program  for 
maintainability  of  electronic  equipment.  For  maintainability 
deaign  guides,  it  refers  to  Navy  Publication  NAVSHIPS  94324. 

The  specification  covers  maintainability  during  desiqn  and  pro¬ 
duction.  It  covers  maintainability  prediction  during  the  pre¬ 
liminary  design  stage.  Maintainability  requirements  are  noted 
for  the  final  desiun  stage,  preproduction  stage  and  during  pro¬ 
duction.  Equipment  Repair  Time  (ERT)  is  used  as  the  measure  of 
mainta  in  'bi 1 i ty . 

The  Appendix  to  specification  MIL  M  23313  (SHIT  S)  covers  "Main- 
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tainability  Design  Evaluation  Procedures"  in  detail.  It  is  speci¬ 
fically  slanted  at  electronic  equipment  and  excludes  mechanical 
hardware  from  the  evaluation  procedures.  Although  the  title  of 
the  Appendix  does  not  indicate  it,  maintainability  prediction 
techniques  are  given  for  the  early  and  late  development  stages  of 
aes ign . 


3  .  QUANTIFICATION  OF  MAINTAINABILITY 

3  .  1  RELIABILITY-MAINTAINABILITY  TRADE-OFFS 

The  selection  of  the  design  approach,  whether  to  use  reliability 
or  maintainability  approaches  to  achieve  the  requ’red  availabil¬ 
ity,  is  based  on  the  functional  requirements  for  the  system. 
Reference  (6)  provides  useful  techniques  in  the  development  of 
reliability-maintainability  trade-offs.  It;  making  the  choice, 
the  following  factors  should  be  kept  in  mind. 

(a)  Even  highly  reliable  systems  will  have  some  failures. 
When  high  reliability  during  short  time  intervals  in  required, 
as  in  the  steering  engines,  high  availability  achieved  through 
reducing  the  MTTR  may  not  be  pertinent,  unless  the  restoration 
is  practically  instantaneous. 

(b)  An  improvement  in  reliability  by  quality  improvement 
(simpler  design,  parts  and  manufacturing  process  control,  etc.) 
will  reduce  the  costs  attributable  to  repairs.  An  improvement 
in  reliability  through  use  of  duplicate  equipments,  each  of 
lower  reliability  will  increase  the  costs  of  maintenance  and 
repair. 

(c)  Equipments  with  low  MTTR  achieved  by  modular  design,  have 
a  tendency  to  increase  the  cost  of  repair.  When  the  low  MTTR 
is  achieved  by  planning  for  main,  -nance  3nd  repair  in  the 
design  phase,  costs  of  repair  tend  to  go  down. 

Reliability  and  Maintainability  in  design  must  be  traded-off  to 
achieve  a  system  or  equipment  design  which  will: 

(a)  Satisfy  a  specified  availability  goal. 

(b)  Satisfy  design  an’  mission  constraints. 


(c)  Result  in  design  optimization  with  respect  to  cost,  per¬ 
formance  and  schedule. 


8-12 


Satisfaction  of  the  mission  goal  --  achievement  of  a  given  level 

of  availability  --  is  determined  by  the  system/equipment  MTTR  and 
MTBF.  MTTR  is  generally  determined  to  a  large  degree  by  (a)  the 
prime  equipment  and  associated  test  equipment  designs  as  they  re¬ 
late  to  the  "on-line  test  approach",  and  (b)  the  packaging  design 
as  :t  relates  to  the  time  required  to  find,  remove  and  replace  a 
failed  element  --  principally  correlated  with  the  "Functional 
level"  of  the  replaceable  element  and  thus  with  the  extent  of  the 
troubleshooting  task  leading  to  correction.  MTBF  is  primarily 
determined  by  the  approach  taken  toward  improving  the  reliabi’ity 
of  the  total  population  of  parts  in  satisfying  the  mission  per¬ 
formance  requirement. 

In  order  to  proceed  deeper  into  thes--'  trade-offs  it  is  necessary 
to  define  the  various  levels  of  performance  or  operational  modes 
for  the  system.  For  a  surface  ship,  one  of  those  might  be  the 
"search"  mode  for  which  we  '  ive  a  specified  operationa1  availabil¬ 
ity  of  0  .90.  Next,  we  must  describe  all  the  equipment,  personnel, 
and  facilities  required  to  support  the  search  mode.  Essentially 
it  means  going  through  the  system  logic  for  each  concept  under 
study  and  using  the  system  model  (discussed  in  Chapter  3)  to  de¬ 
velop  a  failure  effects  analysis  at  the  functional  level.  Since 
functions  relate  to  hardware,  the  following  maintainability  char¬ 
acteristics  can  be  determined:  a)  feasibility  of  performing 
maintenance,  b)  necessity  for  "designing  in"  ease  of  maintenance, 
c)  supporting  hardware  such  as  tools,  test  equipment,  checkout 
gear  required,  and  d)  personnel  required  for  maintenance  and  their 
skill  levels.  The  above  analysis  must  be  performed  in  parallel 
with  reliability  analysis  in  order  to  allocate  availability  to 
the  end  item.  Even  highly  reliable  systems  may  have  an  unaccep¬ 
table  level  of  Availability  if  a  failure  requires  an  excessive 
amount  of  time  to  return  to  satisfactory  operation. 

The  top  level  availability  requirement  can  be  apportioned  among 
the  end  items  required  for  the  search  mode  us inu  standard  relia¬ 
bility  apportionment  techniques  discussed  in  Chapter  f> .  The  Fai¬ 
lure  Effects  Analysis  is  an  aid  in  performing  this.  A  hypothetic;! 
apportionment  is  shown  in  Figure  8-11.  Further  discussion  of  the 
example  will  concentrate  on  the  SONAR  and  ELECTRONICS  enu  item, 
for  which  the  assessed  Availability  goal  is  .^58.  We  are  now 
faced  with  the  task  of  optimizing  the  balance  between  the  Relia¬ 
bility  parameter  (MTBr)  and  the  Maintainability  parameter  (MTTR) . 
Obviously  there  are  a  number  of  trade-offs  as  shown  in  Figure 
8-14,  which  car.  .chiove  the  Availability  requirement  with  an  MTTR 
constraint  cf  .12  days  downtime  failure  (2.88  hrs . )  . 

Now  that,  certain  constraints  have  been  placed  on  the  hypothetical 


SEARCH  MODE  APPORTIONED  AVAILABILITY 


RELATION  OF  MTTR  TO  MTBF 

For  AvaiLa'iility  -  .958 


subsystem,  a  Maintenance  Policy  Study  is  conducted  in  order  to 
determine  what  values  of  MTBF  and  MTTR  seem  to  be  reasonable. 
Concurrently,  reliability  analysis  determines  what  particular 
values  of  MTBF  seem  reasonable  in  light  of  projected  state-of- 
the-art,  development  test  requirements,  costs,  etc. 

Items  which  are  included  in  the  maintenance  policy  study  are: 

(a)  Maintenance  Policy  Study 

1.  Appropriate  echelon  for  repair 

2.  Module  size  determination 

3.  Repair  versus  discard  decisions 

4.  Test  and  checkout  philosophy 

Degree  of  automation 
Inspection  interval 
Special  test  equipment 

5.  Preventive  maintenance  schedule 

6.  Role  of  man  in  system 

Classification/ functions 
Task  definitions 

7 .  Safety  Requirements 

8.  Appropriate  Provisioning  Policy 

(b)  Technician  Requirements 

1.  Selection 

Education 

Experience 

Aptitudes 

Motivation 

2.  Training 

Task  analysis 

Procedures 

Equipment 

Programmed  learning 

3.  Validation  of  Proficiency 

Experimentation 
Man/system  compatibility 
Capabilities  analysis 

(c)  Time  Requirements  for  Corrective  Maintenance 

1.  Localization  time 

2.  Isolation  time 

3.  Disassembly  time 

4.  Interchange  time 
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5.  Reassembly  time 

6.  Alignment  time 

7.  Checkout  time 

8.  Deg”‘^nt ion  factor..  lot  operational  use. 

The  interrelationship  of  these  items  for  Maintainability  is  such 
that  a  change  in  one  will  affect  another  in  terms  ofs  a)  the 
duration  of  system  failure,  b)  the  duration  of  component  failure, 
and  c)  the  ay&tem  cost  initially  and  over  its  lifetime  of  use. 

The  elements  are  compared  in  a  systems  evaluation  model  to  deter¬ 
mine  the  effect  of  the  downtime  of  a  system  on  availability  rela¬ 
tive  to  costs.  Costs  can  be  taken  into  account  by  determining 
manhours  required,  additional  facilities  and  trade-offs  between 
costs  of  items  in  the  supply  pipeline  versus  item  downtime. 

With  the  establishment  of  the  Maintenance  policy,  large  scale 
decision-making  and  trade-offs  are  essentially  complete.  Avail¬ 
ability,  maximum  and  mean  allowable  downtime,  and  minimum  accep¬ 
table  reliability  has-'a  been  assessed  to  the  and  item  level  and 
quantified  for  insertion  into  the  system  specif* cations .  Should 
a  decision  be  made  to  proceed  with  pure  hardware  development, 
these  will  become  design  requirements.  Note  thefc  there  is  still 
some  iatitutde  remaining  for  design  in  that  increased  reliability 
can  be  substituted  for  decreased  maintainability  at  the  black 
box  level. 

In  dealing  with  the  derivation  of  Maintainabi  lity  requirements  the 
man/machine  interface  must  continually  be  evaluated.  In  shipboard 
practice,  the  operator  is  seldom  the  maintenance  technician  and  in 
this  section  the  difference  should  be  distinguished.  First,  we 
are  concerned  with  the  operator  and  his  role  in  system  availabil¬ 
ity.  His  role,  that  of  failure  detection  and  partial  diagnosis, 
is  like  that  of  a  computer  with  many  feedback  loop*.  His  motor 
response  (see  Figure  8-17)  is  a  result  of  how  well  the  machine 
can  tell  him  its  status  during  normal  operation  and  its  troubles 
when  failure  occurs. 

Figure  8-17  also  applies  to  the  maintenance  technician.  That  is, 
there  are  many  alternate  modes  in  which  he  receives  information 
during  the  process  of  system  restoration  (i.e.,  correction  and 
verification).  However,  no  matter  how  well  a  design  is  optimized 
for  man's  sensing  and  cognitive  process,  the  total  job  of  main¬ 
tenance  cannot  be  performed  until  the  physical  constraints  such 
as  space,  and  weight  have  been  overcome  for  hint  to  produce  this 
"motor  response." 
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3.2  ACHIEVEMENT  OF MA INTAINAB ILITY 

All  we  have  done  so  far  is  to  define  the  system,  its  operation 
and  its  design  requirements.  The  process  of  detail  design  now 
gets  underway  and  it  becomes;  Management's  job  to  assure  that 
specific  maintainability  requirements  are  being  met.  The  road¬ 
block  to  maintainability  assurance  lies  in  getting  the  designer 
to  work  with  the  tools  of  Maintainability.  He  must  be  informed 
as  to  what  items  he  must  consider  when  designing  equipment.  It 
is  at  this  point  that  manuals  (7  and  8)  become  a  great  help. 

Achievement  of  maintainability  requires  an  integration  of  the 
maintainability  tasks  into  the  design  cycle.  In  each  step,  the 
maintainability  engineer  is  supporting  the  designer  in  his  effort. 
The  phasing  of  this  support  and  its  integration  into  the  program 
are  indicated  in  Figure  8-19.  Design  toward  required  maintain¬ 
ability  requires  most  of  the  program  aspects  already  discussed 
for  reliability.  Certainly  training  and  indoctr inat ion  of  those 
who  will  ultimately  influence  the  final  product  (designers,  pro¬ 
duction,  etc.)  is  as  important  here  as  in  reliability  achievement. 
Day  to  day  liaison  with  the  designer  will  provide  the  same  rewards, 
understanding  and  acceptance  of  the  discipline.  Participation  in 
Design  reviews  from  concept  to  final  drawing  release  provides  a 
medium  for  training  and  development  of  understanding  in  the  design 
areas.  As  with  reliability,  unless  the  designer  has  a  comprehen¬ 
sible  (to  him)  goal  and  understands  (or  can  be  taught)  the  prin¬ 
ciples  that  will  enable  him  to  achieve  it,  the  effort  of  trying 
to  make  the  equipment  meet  a  maintainability  requirement  is 
fruitless.  A  maintainability  program  commences  during  the  pro¬ 
posal/precontract  study  phase  and  continues  through  design,  de¬ 
velopment,  fabrication,  testing,  and  delivery  of  equipment  to  the 
customer.  Major  program  tasks  include  the  accomplishment  of: 

Design  Analysis  (Liaison)  --  the  systematic  approach  whereby 
maintainability  requirements  are  achieved  effectively  and 
economically  in  the  initial  equipment  design. 

Maintainability  Analysis  --  a  continuing  review  of  the  de¬ 
sign  to  determine  the  degree  of  maintainability  requirements 
incorporated  in  equipment  design. 

Maintainability  Demonstration  --  a  final  hardware  verification 
of  the  actual  degree  of  maintainability  requirements  incor¬ 
porated  in  equipment  design. 


One  facet  of  achievement  of  maintainability  is  the  establishments 
of  these  requirements  and  their  use  as  a  design  control  tool  as 
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presently  specified  in  MIL  M  23313(2). 

In  the  design  to  achieve  a  specified  maintainability  requirement, 
Appendix  to  reference  (2)  describes  maintainability  prediction  by 
the  task  analysis  approach  specified  for  electronic  systems.  The 
requirement  is  specified  as  an  equipment  repair  time  (ERT) .  The 
specification  requirement  is  derived  from  the  equation: 

ERT  (specified)  =  0.37  ERT 

max 

ERTmax  i-s  the  maximum  value  of  ERT  that  should  be  accepted  no 
more  than  10%  of  the  time.  The  factor  0.37  results  from  the  dis¬ 
tribution  assumed  and  assures  a  consumers  risk  of  10%  when  applied 
as  specified. 

Maintainability  prediction  can  be  initiated  in  the  early  develop¬ 
ment  stage,  when  at  least  the  following  have  been  established: 

(a)  The  planned  packaging  arrangement  to  the  extent  that  a 
functional  level  breakdown  into  the  various  equipments,  groups, 
assemblies,  and  subassemblies  can  be  determined. 

(b)  The  planned  diagnostic  procedure  to  the  extent  that  the 
general  levels  of  localization  and  isolation  can  be  determined. 

(c)  The  planned  replacement  method  to  the  extent  that  the 
general  method  of  failure  correction  can  be  determined;  that 
is,  whether  individual  parts,  subassemblies,  assemblies,  or 
units  will  be  replaced  in  making  repairs. 

(d)  The  approximate  quantity  of  various  categories  of  high 
failure  parts  such  as  tubes  and  relays  to  be  included  at  each 
equipment  subdivision. 

ve)  The  level  at  which  normal  equipment  operation  will  be 
confirmed  following  a  repair. 

The  first  step  in  the  procedure  is  to  determine  the  functional 
level  breakdown  of  the  equipnent  or  system.  This  is  done  by 
dividing  the  equipment  or  system  into  its  various  physical  subdiv¬ 
isions  beginning  with  the  highest  subdivision  and  continuing  down 
to  the  items  such  as  parts,  subassemblies,  assemblies  or  units 
that  will  be  replaced  in  corrective  maintenance.  The  functional 
level  breakdown  is  most  easily  established  and  certain  determina¬ 
tions  required  during  the  prediction  are  more  easily  made  if  a 
functional  level  diagram  similar  to  that  shwon  in  Figure  fl-21  is 
prepared.  Here,  a  hypothetical  electronic  equipment  is  subdivided 
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FOR  PARTS  MOUNTED  IN  UNIT 


into  its  various  groups  units,  assemblies,  etc.,  down  to  the 
item  that  will  be  replaced  during  corrective  maintenance.  Each 
block  within  the  diagram  indicates  all  items  having  the  same  main- 
maintainability  features.  For  example  units  repaired  by  replacing 
individual  parts  w'ith  localization  to  the  unit  level,  isolation 
to  the  stage  level,  and  t^st  at  the  group  level  have  been  combined 
and  represented  by  the  "Units"  block  labeled  (a)  in  the  left  hand 
branch  of  Figure  8-21.  Each  branch  of  the  diagram  is  terminated 
with  a  c ircle  which  indicates  the  type  of  item  that  will  be  re¬ 
placed  to  correct  malfunctions  existing  in  that  branch.  The  con¬ 
necting  lines  indicate  maintainability  relations  and  not  electrical 
or  operational  connections.  In  preparing  such  a  diagram  care 
must  be  exercised  in  establishing  the  appropriate  functional 
levels  for  the  various  subdivisions,  especially  where  an  item  may 
have  a  nomenclature  that  includes  the  name  of  one  cf  the  functional 
levels  (for  example,  "Power  Amplifier  Assembly").  In  some  instances, 
the  functional  level  location  of  an  item  may  not  be  the  same  as 
its  nomenclature  indicates. 

After  vhe  functional  level  breakdown  has  been  established  and  the 
functional  level  diagram  prepared,  the  functional  levels  at  which 
localizat ion ,  isolation,  access,  and  test  features  are  applicable 
should  be  determined  based  on  ♦'he  overall  characteristics  of  the 
design.  The  functional  levels  at  which  features  for  local 1 zation , 
isolation,  and  test  are  effective  for  each  replaceable  item  can 
be  indicated  on  the  functional  level  diagram  is  shown  by  the 
symbols.  The  access  functional  level  can  be  determined  directly 
from  the  functional  level  diagram  as  indicated  in  (c)  below,  there¬ 
fore,  a  symbol  identifying  it  is  not  required.  The  functional 
level  at  which  each  of  these  features  is  effective  is  determined 
and  shown  in  the  functional  level  diagram  as  follows: 

(a)  Local i zation.  -  The  functional  level  be  which  a  failure 
can  be  located  without  using  accessory  test  equipment  is 
indicated  by  L, 

(b)  Iso lat ion .  -  The  functional  level  to  which  a  failure  can 
be  located  using  accessory  test  equipment  at  designed  test 
points  is  indicated  by  I. 


Access.  -  The  access  functional  level  for  a  replaceable 
item  is  that  level  to  which  disassembly  must  be  accomplished 
in  order  to  gain  access  to  the  item  that  is  to  be  re¬ 
placed,  and  from  which  reassembly  must  be  accompl i shed 
after  replacement  of  the  item.  This  can  be  determined 
directly  from  the  functional  level  diagram  as  the  func¬ 
tional  level  of  the  first  rectangular  block  above  the 
replaceable  item.  For  example,  to  replace  a  part  in  the 
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left  hand  "Units"  block,  access  must  be  gained  to  the 
unit  level,  and  to  replace  a  subassembly,  access  must  be 
qained  to  the  assembly  level. 

(d)  Tes t .  -  The  highest  functional  level  at  which  restoration 
to  normal  service  can  be  verified  using  s<  If-test  features 
or  other  testing  facilities  is  indicated  by  T. 

The  actual  prediction  is  performed  in  accordance  with  the  follow¬ 
ing  instructions. 

(a)  Calculating  Repair  Times  (R^?.  -  The  repair  t^rr.e  (Rp)  is 

calculated  for  each  category  of  replaceable  item  indicated 
by  a  circle  in  the  functional  level  diagram.  It  is  the 
sum  of  the  maintenance  task  time  intervals  determined 
from  Figure  8-24  in  the  following  manner. 

1.  Loca 1 i zat ion .  -  The  localization  time  interval  is 
determined  by  entering  the  chart  using  the  column 
headed  by  the  type  of  item  that  will  be  replaced 
(indicated  by  a  circle  in  the  functional  level  dia¬ 
gram!  and  continuing  down  this  column  fo  the  row 
with  the  "  Loca  1  i  zat  ion"  column  is  the  value  to  be 
used.  It  the  replacement  items  under  consideration 
are  individually  replaced  pacts  use  the  value  under 
"W"  since  wired  in  parts  normally  out  number  plug- in 
subassemblies,  assemblies  or  units,  use  the  value 
under  "P"  . 
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Isolation.  -  The  isolation  time  interval  is  determined 
in  the  same  manner  as  the  localization  tone  interval 
except  that  the  row  for  entering  the  “isolation" 
column  is  determined  by  the  functional  level  to  which 
isolation  features  are  effective.  This  would  be  the 
level  marked  with  I  ’n  the  appropriate  branch  of 
the  functional  level  diagram.  The  value  indicated  at 
the  intersect  ion  of  this  raw  with  the  "isolation' 
column  is  the  value  to  be  used. 

Access  .  -  The  access  time  interval  is  determined  by 
entering  the  chart  using  column  1  (headed  "Part") 


and  con*  inuiug  down  to  th”  row  dosi  mated  by  toe 
functional  level  to  wh i ch  access  must  be  gained  in 
order  to  perform  the  rcplaomient  tasks.  The  access 
functional  level  for  a  given  replaceable  item,  can  be 
determined  from  the  functional  level  diagram  os  the 
functional  level  of  the  first  rectangular  block  above 
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the  replaceable  item.  The  value  indicated  in  Figure 
8-24  at  the  ir.t  -rsect  i«">n  ;  >t  this  row  with  the  "access" 
column  is  the  value  to  be  used. 


4.  Tes t .  -  The  test  time  interval  is  determined  by 
entering  the  chart  using  column  1  and  continuing 
down  to  the  row  designated  by  the  functional  level 
at  which  restoration  to  normal  service  is  verified. 
This  would  be  the  functional  level  Indicated  by  T 

on  the  functional  level  diagram.  The  value  indicated 
at  the  intersection  of  this  row  with  the  "Test" 
column  is  the  value  to  be  used. 

5.  Interchange .  -  The1  interchange  time  interval  is 
either  0.1  or  0.2  hours  as  shown  in  the  right  hand 
column  of  Figure  8-24.  The  time  indicated  for  sub- 
assemblies,  assemblies,  and  units  (0.1  hours)  apply 
to  these  items  only.  The  time  indicated  lor  parts 
(0.2  hours)  is  applicable  to  all  individually  re¬ 
placed  parts.  The  values  uiveu  for  interchange 
times  are  avei aue  times  for  these  classes  of  items, 
and  include  handling  an  average  amount  of  hardware 
such  as  nuts,  bolts,  and  other  retaining  devices. 

The  repair  time  (Re)  is  the  sum  of  the  t  Lme  intervals 
for  each  tus1'. 


(b)  Calculating  MTTR.  -  After  an  R,,  for  each  circle  of  the 
functional  level  diagram  has  been  calculated,  the  MTTR 
should  be  claculatod  using  the  following  expression: 


MTTR 


K  i  R  -  \  +  K.  R- 

— 1 \ - u i— L- 

K ,  <  K„.  «• 


♦  ...  Km  R  , 

.  .  .  K  r. 


Rs. .  ,  R.  a  ,  . .  .R.  r  ,  are  the  rerun  times  for  re¬ 
placeable  items  hav ing  th«  same  maintainability 
features  (items  within  each  circle  or,  the  func¬ 
tional  1  eve  1  d i a or  am  5 . 


K; ,  Ka ,  * . . Ka ,  are  numbers  which  are  approximately 

proportional  to  the  guant ities  and  re'ative  fro  1- 
ur  e  rate  of  selected  high- failure  rents  -grouped 
within  a  circle  containing  replaceable  items  hav¬ 
ing  the  same  maintainability  features .  The  high- 
far  lure  parts  considered  are  those  Ur  at  will  con¬ 
tribute  tne  majority  of  the  equipment  failure. 


The  we:  gn ting  factors  (K)  are  determined  from  the 
expression : 

11  1 

K  =  NET  +  -y f LTD )  +  ~(KTr )  +  “(NRel)  +  3fNR.es)  +  60  (MMag) 

3  9  3 

+  3 (NCRT)  +  5 (NTT)  (R.l) 

where : 

NET  is  the  number  of  receiving  tubes  in  the  group  of 
items  within  a  circle  containing  replaceable  items 
having  the  same  maintainability  features. 

ND  is  the  number  ef  semiconductor  diodes  in  the  group. 

NTr  is  the  number  of  transistors  in  the  group. 

NRel  is  the  number  of  relays  in  the  group. 

NRC3  is  the  number  of  resolvers  in  the  group. 

NMag  is  the  number  of  magnetrons  in  the  group. 

NCRT  is  the  number  of  cathode  ray  tubes  in  the  group. 

NTT  is  the  number  of  transmitting  and  special  purpose 
tubes  in  the  gr^up. 

The  proportionality  constants  in  express  ton  (8.1),  such 
l  1- 

as  —  and  —  are  approximately  equa1  to  the  average  part 

category  failure  rates  normalized  relative  to  the  average 
failure  rate  for  receiving  tubes. 

In  the  later  stages  of  design,  the  prediction  of  maintainability 
follows  the  same  approach  with  the  exceptions: 

(a)  Tht  functional  levels  are  defined  in  paragraph  4, 

Section  5  of  Reference  (8). 

(b)  Average  part  failure  rates  are  selected  from  Reference 

O')  . 

(c)  Using  a  form  similar  to  Figure  8-27, record  number, 
failure  rates  and  maintenance  task  times  for  individually 
replaced  parts,  replaceable  modular  assemblies  or  units  as 
nppl i cable. 
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The  product  N\  is  the  total  number  of  failures  per  million  hours 
expected  to  be  attributable  to  all  parts  in  the  respective  cate¬ 
gory.  The  sum  of  all  the  "NX"  values  represents  the  total  number 
of  failures  per  million  hours  attributable  tc  the  item  covered  by 
the  worksheet. 

Estimated  times  for  the  maintenance  task  are  taken  from  the  tables 
in  Reference  (2).  The  calculated  repair  time  (Rp)  is  the  sum  of 
times  for  individual  tasks.  The  product  (NXRp)  is  the  total  re¬ 
pair  time  per  million  hours  for  the  category.  The  sum  of  the 
N\RC,  column  represents  the  total  repair  time  per  million  hours 
expected  to  be  required  by  the  item  identified  at  the  top  of  the 
worksheet . 

After  all  worksheets  are  completed,  the  data  should  b-_  consolid¬ 
ated  on  a  summary  sheet  such  as  that  shewn  in  Figure  8-29.  Entry 
of  data  on  the  summary  sheet  and  calculation  of  MTTR  is  as 
follows : 

(a)  List  the  designation  of  the  item  covered  by  each  work¬ 
sheet  in  the  "Item  Designation"  column  of  the  summary  sheet. 

(b)  List  the  sum  of  the  NX’s  from  each  worksheet  in  the  "NX 
Sums”  column  of  the  summary  sheet  opposite  the  respective  item 
designation. 

(c)  List  the  sum  of  the  NXR's  from  each  worksheet  in  the 
"NXRp  Sums"  column  of  the  summary  sheet  opposite  the  res¬ 
pective  item  designation. 

(d)  Record  the  totals  for  the  "NX  Sums"  column  and  the 
"NXRp  Sums"  column  at  the  bottom  of  the  respective  columns. 

Calculate  the  predicted  MTTR  as  indicated  at  the  bottom  of 
the  summary  sheet. 

3.3  PREDICTION  FOR  MECHANICAL  SYSTEMS 


The  approach  to  prediction  of  mechanical  systems  is  essentially 
identical.  Prediction  of  repair  times  can  be  initiated  when: 

(a)  The  design  has  progressed  to  the  point  that  the  major 
parts  are  determined. 

(b)  The  planned  replacement  method  can  be  determined;  that 
is,  whether  parts,  components,  assemblies  or  units  will  be 
replaced  in  making  repairs. 
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(c)  The  maintenance  schedule  is  established. 

The  prediction  begins  with  the  determination  of  the  functional 
level  breakdown  of  the  system.  A  functional  level  diagram  should 
be  used  for  completeness .  Parts  +hat  would  be  replaced  simul¬ 
taneously  should  be  treated  as  a  single  assembly.  Where  different 
modes  of  failure  (see  Chapter  12)  will  cause  a  difference  in  the 
repair  or  restoration  task,  they  should  be  treated  as  separate 
parts . 

Using  a  worksheet  similar  to  Figure  8-29, record  the  replaceable 
units  (part,  assembly  or  component  level).  Determine  the  ex¬ 
pected  failure  rate  of  the  part  (by  failure  modes  if  applicable) , 
assuming  replacement  of  parts  as  prescribed  in  the  maintenance 
schedule . 

Compute  the  products  NX  as  before,  add  and  record  in  the  space 
provided . 

For  each  listed  unit,  estimate  (as  if  you  were  planning  the 
repair  job)  the  length  of  time  necessary  to  perform  each  step 
of  the  task.  The  normal  steps  in  many  mechanical  repairs 
follow  a  pattern,  such  as: 

(a)  Diagnosis  of  trouble  (localization) 

(b)  Isolation  and  cool  down  (isolation) 

(c)  Removal  of  obstructions  (access) 

(d)  Disassembly  (access) 

(e)  Repair  or  replace  parts,  including  fittina,  alignment, 
balance,  etc.  (replace) 


(f) 

Reassembly  (access) 

(9) 

Replacement  of  obstructions  (access) 

(h) 

Restore  normal  conditions  (purify,  flush,  etc.) 

(align) 

(i) 

Test  (test. 

Compute 

the  repair  time  Rp  ky  summing  the  time 

for  the 

individual 

steps . 

Where  steps  would  normally  be  performed 

concurrently,  the 

time  for  the  combined  operation  should  be  estimated  and  recorded. 


A 


Compute  and  sum  the  NARr  terms.  The  MTTR  (inherent)  is  determined 
as  before  from  the  equation: 

l  N'Rr 

MTTR  =  — 

/  N' 

( . . 

3.4  APPLICATION  OF  PREDICTION 

The  prediction  of  maintainability  is  used  as  a  diagnostic  tool. 

The  MTTR  just  computed  provides  only  part  of  the  answer,  the 
average  restoration  time.  Where  this  time  is  excessive,  the 
design,  planned  replaceable  unit  level,  access  or  other  factors 
must  be  improved.  When  an  ERTmax  has  been  required,  an  estimate 
of  compliance  can  be  obtained  by  summing  all  of  the  part  entries 
for  NARp  that  exceed  the  prescribed  value,  dividing  this  sum  by 
the  total  sum  of  N\RP  's  if  the  fraction  exceeds  0.10,  the  equip¬ 
ment  fails  to  comply. 

The  prediction  is  a  useful  tool  i,»  determining  where  best  to 
apply  the  techniques  for  improving  maintainability.  Like  relia¬ 
bility,  maintainability  can  only  be  derived  through  sound,  inher¬ 
ent  design.  The  designer  can  only  do  this  by  an  awareness  of 
what  the  problems  are  and  what  tools  exist  to  solve  the  problems. 
True,  he  should  be  aware  of  the  quantitative  downtime  requirements 
which  his  system  or  equipment  must  meet,  but  he  also  needs  the 
knowledge  of  1)  experienced  specialists  in  maintenance  analysis, 
and  2)  human  factors  types  and  those  who  deal  with  the  life 
sciences.  In  addition,  handbooks  such  as  NAVSHIPS  94324  should 
be  made  available  to  each  design  group.  Much  of  the  knowledge 
in  these  areas  eliminates  guesswork  and  gives  the  designer  a 
firm  basis  for  which  to  package  the  subsystem  (end  item)  within 
the  ship,  the  components  within  the  subsystem  and  the  parts 
within  the  components. 

One  of  the  more  widely  used  techniques  in  industry  is  the  Design 
Checklist,  in  conjunction  with  design  reviews.  Design  checklists 
can  be  used  to  convert  quantitative  human  engineering  require¬ 
ments  into  a  qualitatively  good  design. 

During  the  design,  the  designer  should  have  before  him  contin¬ 
uously,  the  objective  of  making  the  equipment  easy  to  maintain. 
Reference  (8,  9  and  .10)  give  comprehensive  coverage  of  the  design 
for  maintainability  of  electronic  equipment.  Far  less  exhaustive 
are  the  details  provided  for  designers  of  mechanical  systems. 

Since  mechanics  are  about  the  same  3ize,  build  and  strength  as 
electronics  technicians,  many  of  the  same  rules,  given  in  refer¬ 
ence  (8),  apply.  Reference  (11)  provides  additional  descriptions 
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of  the  capability  of  operating  and  maintenance  personnel. 

Decisions  as  to  the  division  of  requirements  among  the  seven 
major  elements  of  downtime  requires  detailed  cost  studies.  If 
we  assume  that  meeting  system  downtime  requirements  means  that 
we  in  fact  do  meet  specified  mission  requirements,  then  the  alloca¬ 
tion  problem  becomes  strictly  an  economic  one  --  i.e.,  attaining 
the  specified  downtime  capability  at  minimum  cost. 

As  an  example,  one  of  the  primary  means  for  reducing  system  down¬ 
time  is  to  substitute  automatic  checkout  and  diagnostic  equipment 
for  the  slower  human  operator  and  to  use  modularized,  plug  in 
components.  Decisions  to  automate  are  generally  made  on  a  systems 
basis-  the  level  to  which  automation  would  be  carried  (e.g., 
automatic  fault  isolation  to  a  replaceable  module  level)  would 
be  determined  by  cost  tradeoff  studies.  Results  would  establish 
consistent  guide  lines  for  allocating  downtime  requirements  below 
system  level  to  the  subsystem  and  the  component  level. 

The  most  time-consuming  element  of  downtime  is  generally  diagnosis. 
However,  substitution  of  automatic  diagnostic  equipment  and  proper 
selection  of  the  module  size  to  which  the  failure  will  be  isolated 
can  reduce  this  time  almost  to  zero.  The  same  holds  true  for 
detection  and  verification,  depending  on  the  extent  of  automation 
desired.  This  leaves  the  “correction"  element  as  the  one  which 
ultimately  becomes  the  most  limiting  factor  in  this  example. 

Cost  tradeoffs  might  indicate  that  it  would  be  cheaper  to  go  to 
redundant  switch-in  spares,  thus  reducing  corrective  time  and 
eliminating  some  of  the  automatic  checkout  features.  This  is 
but  one  example  of  the  many  tradeoffs  necessary  to  determine  the 
least  cost  configuration  and  to  define  support  items  necessary. 

The  following  characteristics  (among  others)  affect  the  ease  and 
rapidity  of  repair: 

(a)  Accessibility,  including  room  to  operate  tools  required. 

(b)  Clarity  of  instructions  and  diagrams. 

(c)  Marking  and  identification 

(d)  Displays,  gauges,  and  controls. 

(e)  Weight,  including  provision  of  handling  gear  &  lifting  pads, 
(fl  Interchangeability. 

(g)  Proper  tools. 

(h)  Visibility. 

The  cost  studies  involved  in  tradeoffs  should  include,  but  not 
be  limited  to: 
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(a)  Cost  of  parts  or  modules. 

(b)  Cost  or  value  of  salaries  of  repair  personnel. 

(c)  Cost  of  training  or  repair  personnel. 

(d)  Cost  of  rework  of  modules  at  the  factory,,  or  repair 

activity  (tender  or  shipyard) . 

(e)  Administrative  costs  of  procurement,  storage  and  shipping. 

(f)  Costs  of  diagnostic,  test  and  repair  tooling. 

3.5  MAINTAINABILITY  PROGRAM  CONSIDERATIONS 


But  prediction  of  maintainability  is  only  one  of  the  phases  of 
the  total  program.  Maintainability  achievement  requires  the 
same  comprehensive,  across  the  board  consideration  as  reliability. 
Figure  8-34  indicates  the  areas  of  proqram  application,  showing 
the  interplay  with  design,  reliability,  logistic  planning,  train¬ 
ing,  software  and  support  requirements. 


4. 


MAINTENANCE  ANALYSIS 


I 


*4 


Like  system  analysis,  maintenance  analysis  is  an  iterative  pro¬ 
cess  that  appears  to  be  gigantic.  It  is  a  tedious  process  but 
it  is  controllable.  In  the  Conceptual  Phase  of  a  project,  gross 
statements  based  on  experience  have  to  be  made  for  performance, 
maintainability,  availability,  etc.  This  is  so  because  the  con¬ 
ceptual  phase  of  a  project  is  a  wish  fo  develop  something  novel 
based  on  past  experience.  The  past  experience  is  the  life  saver 
that  makes  it  possible  to  make  gross  statements  with  a  fair  degree 
of  accuracy.  Past  experience  also  gives  industry  the  incentive 
to  propose  novel  projects  or  bid  on  them.  Through  continuous 
iterative  analysis,  the  original  gross  statements  evolve  into 
workable  hardware  systems  that  meet  performance  requirements. 

4.1  INTEGRATED  MAINTENANC E  MANAGEMENT 

The  integrated  maintenance  management  concept  (3)  includes  docu¬ 
mentation  in  the  form  of  Maintenance  Engineering  Analysis  Records 
(MEARS)  to  control  the  data  needed  f,>r  maintenance  analysis.  See 
chapter  17 . 

The  maintainability  of  the  design  is  documented  as  follows: 

1.  The  maintenance  concept  has  beer,  reviewed  and  confirmed. 


2.  The  contractor's  q”alitative  maintainability  design 
features  of  the  product  have  been  verified. 


MAINTAIN  ABILITY  PROGRAM 


Design  Analysis 
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3.  The  maintenance  requirements  and  tasks  established  for 
the  article  have  been  demonstrated. 

4.2.  MAINTENANCE  CONCEPT 

Analysis  of  a  Shipboard  Diesel  Generator  is  used  tc  shoo  how  a 
maintenance  analysis  is  done.  We  have  picked  a  3000  brake  horse¬ 
power  diesel  which  has  a  fuel  consumption  of  0.42  pounds  per  brake 
horsepower  hour.  Two  power  plants  are  used  to  assure  a  continuous 
supply  of  power.  If  we  assume  the  ship  will  have  a  three-month 
mission,  then  approximately  419,000  gallons  of  diesel  rue]  will 
be  consumed  on  the  mission.  It  is  not  practical  to  carr ,  such  a 
quantity  of  fuel  on  board  ship  so  arrangements  have  to  made 
for  resupply  from  tenders.  This  means  piping  has  to  be  installed 
for  taking  the  fuel  from  the  tender  to  the  diesel  fuel  tanks  in 
the  ship. 

Various  kinds  of  maintenance  and  operations  will  be  required  to 
keep  at  least  one  diesel  generator  operating  at  all  times.  A 
90-day  mission  amounts  to  2160  hours  of  running  time  which  can 
be  evenly  distributed  to  each  diesel  for  1080  hours  each.  If  the 
diesel  generators  are  good  for  3000  hours  of  operation  between 
overhaul,  the  overhaul  plans  have  to  be  made  to  do  this  after 
every  third  mission. 

Instructions  have  to  be  provided  to  tell  the  technicians  how  to 
switch  over  from  one  generator  to  the  other  periodically.  There 
will  be  light  maintenance  work  required  to  keep  the  diesels  run¬ 
ning  efficiently.  Air  and  fuel  filters  will  have  to  be  cleaned 
periodically  to  remove  accumulated  carbon  deposits.  The  crank¬ 
case  oil  has  to  be  checked  and  replenished. 

The  operating  generator  has  to  be  monitored  to  assure  that  the 
electrical  output  is  consistent  with  requirements  of  the  ship¬ 
board  equipment.  Thi3  information  has  to  be  provided  as  in¬ 
structions  for  the  technicians  that  do  the  work.  Brushes  on  the 
generators  have  to  be  inspected  periodically  to  assure  they  are 
not  worn  out.  Since  the  brushes  can  wear  out,  provisions  have 
to  be  made  to  stock  spare  brushes  on  the  ship. 

A  detail  analysis  has  to  be  made  of  the  diesel  and  generator  to 
determine  which  parts  will  have  to  be  replaced  at  the  3000-hour 
overhaul  period.  This  analysis  will  be  based  on  experience  with 
exist- ing  diesel  generutors  or  i  f  it  is  a  new  design,  experience 
with  similar  designs  will  be  used.  The  detail  analysis  will 
develop  data  for  use  in  overhaul  manuals,  provide  a  list  of 
repair  parts  for  an  overhaul  operation,  oevelop  a  list  of  tools 
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needed  for  an  overhaul  and  an  estimate  of  the  manhours  needed  tc 
do  an  overhaul. 

A  decision  has  to  be  made  about  where  the  overhaul  work  is  to  be 
done.  a  3000  horsepower  diesel  engine  weighs  approximately  207,000 
pounds  so  it  will  x'-*  ’ogical  to  do  the  overhaul  on  board  ship. 
Maintenance  analysis  will  determine  who  ^11  do  the  overhaul  work} 
the  ship's  crew  or  repair  specialists  from  shore  installations. 

If  shore  specialists  are  to  be  used,  then  missions  have  to  be 
scheduled  so  the  ship  will  be  at  the  right  location  when  overhaul 
work  is  done. 

While  analyzing  the  diesel  generator  for  overhaul  maintenance,  the 
personnel  relation  to  the  equipment  will  be  analyzed.  Most  likely, 
hoisting  systems  will  be  installed  over  the  diesels  to  assist  in 
disassembly  of  the  heavy  parts  such  as  the  cylinder  head  and  the 
pistons.  If  the  analysis  is  done  early  in  the  ship  design  it  will 
be  possible  to  include  qualitative  maintainability  characteristics 
in  the  total  design  like  the  Design  Work  Study  Program  does. 

A  plan  has  id  be  made  for  phasing  the  operation  of  the  diesels  so 
chat  both  diesels  do  not  have  to  be  overhauled  at  the  same  time. 

It  would  be  logical  to  run  one  diesel  a  thousand  hours  before 
starting  the  other.  Then  diesel  operation  could  be  switched 
every  week  or  after  168  hours  of  operation.  One  thousand  hours 
of  operation  would  occur  after  42  days  in  a  mission. 

Maintenance  plans  of  diesel  generators  have  to  consider  all  the 
ships  of  the  same  design  using  the  equipment  to  assure  that,  all 
repair  parts  are  ordered  at  the  same  time.  Appreciable  savings 
in  unit  costs  can  be  made  by  ordering  the  maximum  number  of 
repair  parts  at  one  time. 

The  maintenance  analysis  has  to  be  done  to  assure  that  mainten¬ 
ance  can  bo  dene,  that  repair  parts  will  be  available,  that 
technical  information  is  available  for  the  technicians  and  that 
tools  and  test  equipment  will  be  available 

4.3  MAINTAINABILITY  TASK  ANALYSIS 

A  completed  maintainability  task  analysis  is  shown  to  demonstrate 
the  analysis  of  the  steps  in  repairing  a  faulty  control  and  in¬ 
dicator  panel  in  a  Test  Station.  The  sequence  starts  with  dis¬ 
assembly  to  gain  access  and  isolating  the  defective  part  (Fioure 
8-3"’).  Then  in  figure  '-38  the  steps  in  making  the  re- air  are 
defined.  This  type  of  analysis  provides  the  basic  planning  in¬ 
formation  necessary  to  provide  the  proper  personnel,  tools. 
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supply  support  and  personnel  training  requirements  for  later  use 
of  the  equipment  aboard  ship. 


5  .  MAINTAINABILITY  DEMONSTRATION 

5.1  ELECTRONIC  SYSTEMS 

As  with  reliability  or  performance,  a  requirement  for  maintain¬ 
ability  's  unenforceable  unless  some  means  of  verification  is 
provided.  For  electronics  equir~ent  MIL  M  23313A(2^  provides  a 
maintainability  demonstration  plan  to  be  applied  to  a  preproduc¬ 
tion  model  before  the  start  of  production.  The  test  consists  of 
inducing  failures  in  the  equipment,  requiring  a  technician  to 
identify  and  repair  the  failure  without  prior  knowledge  of  the 
failed  item.  For  the  test  plan  of  Reference  (2),  twenty  failures 
are  selected  in  rough  proportion  to  their  probability  of  occur¬ 
rence.  The  failed  component  is  selected  using  random  numbers  or 
other  unbiased  techniques.  The  time  for  each  step  of  the  repair 
is  measured  and  recorded.  The  time  is  adjusted  for  the  experience 
level  of  the  technician  by  a  factor  based  on  his  years  of  experi¬ 
ence  . 

When  the  twenty  repairs  have  been  completed,  the  evaluation  is 
conducted  as  follows : 

The  acceptance  criterion,  log  MTTRG  «  log  ERT  +  0.397 (S),  assures 
a  probability  of  .95  of  accepting  an  equipment  or  systems  as  a 
result  of  one  test  when  th^  true  geometric  mean-time-to-repair  is 
equal  to  the  specified  equipment  repair  time  (that  is,  a  probabil¬ 
ity  of  0.05  of  rejecting  an  equipment  cr  system  having  a  true 
MTTRq  equal  to  the  specified  ERT).  This  was  derived  by  using 
conventional  methods  for  establishing  acceptance  criteria  (Chap¬ 
ter  11) .  The  conventional  methods  for  determining  acceptance 
based  on  the  measured  mean  of  a  small  sample,  that  is,  sample 
size  less  than  30) ,  ar 1  when  the  true  standard  deviation  (a)  of 
the  population  can  only  be  estimated,  is  to  compare  the  measured 
mean  with  the  desired  mean  using  the  expression: 


t 


8.2 


where :  S 


cr  the  standard  deviation  of  the  sample? 


i 


i 
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the  sample  or  measured  mean 
:  the  specified  or  desired  mean 
the  sample  size 

:  the  value  of  one  measurement  of  the  sample. 


the  decision  to  accept  the  product  will  be  made  when  the  test 
results  give  a  value  of  t,  as  calculated  from  expression  8.2 
numerically  less  than  or  equal  to  a  value  of  t  obtained  from 
'‘Student's  t"  distribution  tables  at  the  established  level  (that 
is,  0.99,  0.95,  0.90,  and  so  forth)  of  acceptance  and  the  appro¬ 
priate  sample  size.  The  "Student's  t"  distribution  tables  (for 
a  single  tailed  area)  give  a  value  of  t  =  1.729  at  the  0.95  ac¬ 
ceptance  level  when  the  sample  size  is  20  (that  is,  19  degrees 
of  freedom.)  The  table  for  single  tailed  area  is  used  since 
only  values  of  MTTRq  greater  than  the  specified  ERT  are  critical. 
An  equipment  with  any  value  of  MTTRq  lower  than  the  specified 
ERT  is  acceptable.  To  apply  expression  8.2  to  the  maintainabil¬ 
ity  test,  let  x,  =  log  ERT  (speci fied) , x  =  log  MTTRq  (measured), 


S  =  the  measured  standard  deviation  of  the  loaarithms  of  th 


sample  of  measured  repair  the,  and  N  =  the  sample  size  of  20. 
The  measured  MTTRq  is  then  compared  with  the  desired  ERT  by  cal¬ 
culating  the  value  of  t  using  the  expression  below: 


_  (log  mttrg  -  log  ert)  ig 

s 


The  equipment  under  test  can  be  accepted  if  the  value  of  t  cal¬ 
culated  from  expression  8.3  is  equal  to  or  less  than  +1.729  (the 
value  of  t  from  the  "Student's  t"  distribution  tables  at  an 
acceptance  level  of  .95  when  the  sample  size  is  20).  Therefore, 
the  equipment  should  be  accepted  when: 


/ —  (log  MTTRq  -  log  ERT) 

V 19  - - - - - - - -  6  +1.729 

S 


Upon  rearranging  and  simplifying  this  expression,  the  acceptance 
criterion  is  obtained  as  shown  below: 


log  MTTRq  -  log  ERT  * 


1.729(S) 


log  MTTRg  *  log  ERT  +  .397 (s) 


•V 


iVd-l 
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In  the  event  the  criterion  is  not  met,  the  test  shall  be  repeated. 
If,  for  the  second  test,  the  criterion  is  met,  the  maintainabil¬ 
ity  requirement  for  the  preproduction  model  will  be  considered  to 
have  been  met.  If,  for  the  second  test,  the  criterion  is  still 
not  mot,  the  equipment  will  be  considered  to  have  failed  the 
maintainability  requirements  for  the  preproduction  model. 

Therefore,  the  combined  probability  of  acceptance  of  an  equipmen*- 
or  system  with  a  true  MTTRq  equal  to  the  specified  ERT  is  0.S5 
+  0.05  (0.95)  or  0.9975.  Thus,  equipment  of  specification  qual¬ 
ity  (that  is,  MTTRq  =  ERT)  or  better  will  almost  certainly  pass 
the  combined  test. 

The  test  procedure  is  designed  for  producers  risk  of  .0025  and 
consumers  risk  of  .0975  based  on  the  assumption  of  log  normal 
distribution  of  times  to  repair  and  the  specified  ERT. 

5.2  MECHANICAL  SYSTEMS 


Again  the  demonstration  of  compliance  is  necessary  if  assurance 
of  the  achievement  of  the  requirement  is  to  be  obtained.  While 
there  is  less  evidence  in  mechanical  repairs  than  in  electronic 
systems  to  indicate  that  repair  times  are  distributed  log-nor¬ 
mally,  the  underlying  distribution  can  be  estimated  from  the 
maintainability  prediction  data.  The  design  of  a  test  procedure 
for  acceptance  testing  will  be  done  in  the  manner  discussed  in 
Chapter  11.  Again,  Where  a  test  proposed  by  a  contractor  is 
being  reviewed,  or  a  test  being  proposed  to  a  contractor,  obtain 
the  assistance  of  a  qualified  statistician. 


6.  APPLICATIONS  TO  CURRENT  WORK 

6.1 _ DEFINITION  OF  REQUIREMENTS 


We  have  just  run  through  a  multitude  of  pages  in  an  attempt  to 
show  you  the  basic  ingredients  of  Maintainability,  how  it  gets 
into  the  systems  development  process  and  how  it  relates  to  other 
design  parameters.  Most  of  you  are  doing  engineering  work  on  new 
or  modified  subsystems  Which  eventually  will  be  installed  in 
existing  ships.  You  are  probably  aware  of  the  present  operational 
support  problems  which  now  exist  in  similar  equipment  and  are 
desirous  to  get  maintainability  built-in  to  new  specifications 
on  an  equal  level. 

There  are  two  items  which  have  to  be  answered:  how  much  Main¬ 
tainability  is  needed,  and  how  sure  do  you  want  to  be  that  your 
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requirement  is  met?  To  answer  the  former,  it  is  necessary  first 
to  know  the  mission  of  the  equipment,  i.e.,  how  often  is  it 
demanded  for  operation,  who  will  operate  it  and  what  is  its  en¬ 
vironment.  Second,  the  amount  of  Maintainability  needed  depends 
on  cost  of  achieving  various  levels  of  Maintainability  versus  cost 
savings  through  reduced  down  time.  The  only  reason  for  this  re¬ 
latively  new  discipline  is  reduced  costs  with  an  attending  in¬ 
crease  in  system  effectiveness. 

Next,  what  confidence  is  wanted  that  the  MTTR  will  be  met?  This 
again  relates  to  cost.  It  also  relates  to  the  sensitivity  of 
Maintainability  of  individual  elements  to  the  overall  system 
availability.  Defining  requirements  becomes  of  question  of  how 
much  it  costs  to  produce  versus  how  much  it  costs  to  use  for 
various  levels  of  availability. 

6.2  CONTRACTING  FOR  MAINTAINABILITY 


In  order  that  the  contractor  understands  fully  the  BUSHIPS  need 
for  Maintainability  it  must  be  clearly  and  explicitly  defined  in 
contracts  for  new  equipment  or  in  follow-on  contracts  for  modi¬ 
fications  of  existing  equipment.  Specifically,  the  following 
items  should  be  considered  for  inclusion  in  a  hardware  contract: 

a.  Quantified  MTTR  or  availability  goals  for  consideration 

by  the  contractor:  The  word  "consideration"  is  used  since 
it  is  desirable  to  give  the  contractor  some  latitude  in 
order  that  he  may  analyze  entative  goals  and  perhaps 
submit  a  recommendation  to  BUSHIPS  as  to  how  goals  could 
be  changed  for  reduced  costs.  (But  do  specify  a  maximum 
allowable  down  time  or  ERT  where  appropriate) . 

b.  Tell  the  contractor  how  much  you  expect  Maintainability 
to  be  weighted  in  his  total  design  effort  against  other 
technical  disciplines  such  as  the  various  aspects  of 
performance,  reliability,  etc. 

c.  If  contract  is  of  the  CPIF  type,  specify  how  incentive 
fees  will  be  paid  on  the  basis  of  the  contractors  perform¬ 
ance  in  the  maintainability  portion  of  the  Dependability 
Plan. 

d.  Tell  the  contractor  what  special  maintainability  problem 
areas,  if  any,  to  investigate. 

e.  Supply  in  the  contract  specification,  other  applicable 
documents  such  as  the  general  specificat^'  i  MIL  M  23313. 
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Tell  the  contractor  which  portions  of  the  specifications 
are  applicable  to  your  particular  subsystem.  Delineate 
the  maintenance  analysis  forms  he  must  use  and  the  fre¬ 
quency  of  reporting  results. 
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Chapter  9 
DATA  ACQUISITION 

In  the  development  of  a  system  or  equipment  a  1 .rgr  amount  ot 
data  is  collected  and  used.  The  data  adds  to  tne  objective  evid¬ 
ence  used  by  the  designer  in  making  decisions  about  the  des:u_,  ', 
by  the  inspector  in  making  decisions  about  acceptance  ,-nd  the 
program  manager  in  making  decisions  concerning  the  program.  In 
this  chapter  we  will  attempt  to  develop  two  areas  : 

(1)  The  interpretation  of  data;  that  is,  the  recognition 
that  a  particular  group  of  data  :s  a  sample  drawn  from  a  larg 
population,  and  that  inferences  can  be  made  about  the  c  *ct 
istics  of  that  popt  lation  from  the  sample. 

(2)  The  generation  or  collec  ion  of  data,  useful  sources, 
limitations  on  their  validity  and  practical  value  of  use  data 

In  the  design,  procurement,  te.  irg  and  operational  use  of  an 
equipnent  there  is  an  infinite  va  iation  in  character istics .  N  • 
two  equipments  are  ever  the  same.  If  you  put  two  identical1' 
equipments  on  test  they  won't,  fail  at  the  same  time.  The  differ¬ 
ences  are  each,  in  most  cases,  minor  -  a  half  a  thousandth  here 
in  clearance,  a  tenth  of  an  inch-pound  in  balance,  an  almost  un¬ 
detectable  difference  in  roughness  of  surface  funish  and  so 
forth.  Within  crude  limits,  the  equipments  are  identical.  With 
more  exact  measurements,  a  difference  can  be  found. 

Because  of  this  variability  from  sample  to  sample,  the  interpret¬ 
ation  of  the  data  depends  on  the  branch  of  scientific  method 
called  statistics. 

The  real  utility  of  sample  data  from  tests  of  individual  units 
lies  in  the  capability  of  making  statements  about  the  population 
from  which  the  sample  was  taken.  Mistakes  can  be  made  when  you 
try  to  infer  a  general  rule  (make  a  statement  about  the  populat¬ 
ion  parameters)  from  a  specific  case  (the  sample  data).  Statis¬ 
tical  theory  is  the  only  method  now  known  t..at  permits  some  degree 
of  control  to  prevent  such  mistakes. 

1.  THE  POPULATION 

Before  any  specific  functions  are  discussed,  it  is  important  to 
state  the  usual  nature  of  the  statistical  problem  in  order  to 
explain  how  one  should  view  the  formulas  and  curves  to  be  present¬ 
ed.  Collected  data  received  from  any  testing  or  surveillance 
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program  should  be  viewed  as  information  to  be  used  in  guessing 
what  will  happen  in  the  future  --  that  is,  in  prediction.  Of 
course,  there  is  an  element  of  historical  interest  in  knowing 
what  did  happen  just  for  the  sake  of  knowledge  itself.  The  real 
payoff,  however,  lies  in  using  the  data  as  a  sample  from  some 
population,  and  the  job  at  hand  is  one  of  describing  this  pop¬ 
ulation  from  some  of  the  characteristics  of  the  sample  data. 

For  example,  one  would  like  to  be  able  to  take  field  data  on 
failures  in  a  specific  weapons  system  and  write  the  formula  for 
the  failure  density  of  all  future  failure  experience  which  will 
be  met  with  this  same  system  and  even  with  improved  versions  of 
the  system,  using  adjustment  for  the  system  modifications. 

The  moral  of  this  view  is  that  field  data  constitutes  a  sample, 
that  random  sampling  peculiarities  must  be  smoothed  out,  that 
population  density  parameters  must  be  estimated,  that  the  esti¬ 
mation  errors  must  themselves  be  estimated,  and  --  what  is  even 
more  difficult  --  that  the  very  nature  of  the  population  density 
must  be  estimated.  To  achieve  these  ends,  it  is  necessary  to 
learn  as  much  as  possible  about  the  possible  population  density 
functions,  and  especially  what  kind  of  results  we  can  expect 
when  samples  are  drawn,  the  data  are  studied,  and  we  attempt  to 
go  from  data  backward  to  the  population  itself.  It  is  also 
important  to  know  what  types  of  population  densities  are  produced 
from  any  given  set  of  engineering  conditions.  This  implies  the 
necessity  for  developing  probability  models,  or  going  from  a  set 
of  assumed  engineering  characteristics  to  a  population  density. 

it  is  customary,  even  necessary,  in  statistical  analysis  to 
develop  from  the  physical  engineering  principles  the  nature  of 
the  underlying  distribution.  The  sample  of  data  is  then  tested 
^gainst  the  assumed  distribution. 

Tie  usual  parameter  of  interest  in  reliability  is  the  distribution 
of  times  to  failure,  called  the  probability  distribution  function 
or  failure  density  function.  The  failure  density  function  may  be 
discrete,  that  is,  only  certain  (integral)  values  may  occur,  as 
in  tests  of  an  explosive  squib.  Success  or  failure  will  occur 
on  anv  trial,  time  not  being  considered.  Or  it  may  be  continuous, 
any  value  of  time  to  failure  being  possible. 

In  the  analysis  of  parameters  of  populations  the  following  dis¬ 
tributions  have  been  found  to  be  useful. 

1.1  BINO.IAL  DISTRIBUTION 

The  Binomial  distribution  arises  from  a  series  of  Bernoulli  trials. 
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A  Bernoulli  sequence  of  trials  .s  defined  as  sequence  of  experi¬ 
ments  satisfying  the  following  conditions* 

1.  For  each  experiment,  the  result  is  either  success  or 

failure? 

2.  The  probability  of  success  is  the  same  for  every  experi¬ 
ment; 


3.  Each  trial  is  independent  of  all  others, 


The  binomial  failure  density  function  is 


f(r)  =  (“) 


n-r  r 

P  q  = 


n: 


(n-r)  I  r  1 


n-r  r 

p  q 


where 


n 


is  the  number  of  trials 


is  the  number  of  failures 


p  is  the  probability  or  success 

q=*(l-p)  is  the  probability  of  failure. 

Hence  f(r)  is  the  probability  of  exactly  r  failures  out  of  n 

when  the  probability  of  a  success  is  p. 

The  probability  of  r  or  more  failures  out  of  n  Bernoulli  trials 
is  given  by: 

n 

•  T  ,n,  n-i  i 
F(r»  -  L  (f)  P  q 

i=r 


F(r)  is  the  cumulative  distribution  function,  and  may  be  inter¬ 
preted  as  the  probability  of  r  or  more  failures  out  of  n  trials. 

Since  the  equipment  must  either  succeed  or  fail,  the  sum  of  the 
probabilities  equals  unity. 

The  probability  of  success,  R(r),  where  success  is  defined  as 

less  than  r  failures,  is  the  complement  of  F(r),  that  is: 

n 

,  .  .  ,  >  .  F  ,  n »  n  —  i  i 

R(r)  *  1  -  F(r)  *  1  '  ^  ^ i  P  q 

i*r 
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We  can  define  the  expected  nunber  of  successes,  E(s)  as  the 
average  or  mean  value  of  the  distribution.  This  value  is  the 
product  of  the  number  of  trials  and  the  probabilitv  of  success 
on  each  individual  trial.  That  is: 

E(s)  =  np 

and  the  variance  of  s  (number  of  successes)  is: 

Var  (s)  =  a*  =  npq 

Hence,  the  standard  deviation  n  is: 

s 


The  independent  parameter  of  the  Binomial  is  p,  the  probability 
of  success.  The  properties  of  the  distribution  are  shown  in 
Figure  9-6. 

Basically,  the  Binomial  distribution  is  utilized  in  cases  where 
the  equipment  operates  in  definite  cycles  such  as  an  on-off 
switch,  or  in  cases  w>.  to  the  cycle  is  some  minutes  in  length  but 
involves  varying  stresses  and  operation  and:  hence,  varying  pro¬ 
babilitv  of  failure  from  minute  to  nunute,  provided  each  trial  is 
independent,  success  or  failure  on  any  individual  trial  not 
af'r'cting  the  results  of  prior  or  subsequent  trials,  and  each 
overall  trial  has  the  same  probability  of  success.  An  example 
would  be  a  missile  flight.  Such  flights  are  programmed  so  that 
each  flight  is  a  duplicate  of  the  others.  Ihe  same  stress-time 
cycle  is  imposed. 

1  .2  EXPO  HE  NT  IAL  D I STR  TBlrT  ION 

The  Exponential  failure  density  function, 

f  ( t )  V  c  ' t 

X  -  mean  failure  rate 

t  -  time  under  consideration 

is  widely  used.  The  distribution  is  a  special  case  of  both  the 
Woibull  and  the  Gamma  distributions. 

The  equation  above  is  the  one  most  usually  thought  of  when  the 
Exponential  distribution  is  spoken  of.  This  is,  of  course,  the 


)  F  FAILUR] 
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Expor»3ntial  distribution  cf  failures  over  time.  Since  R(t)  is  the 
probability  of  no  failure  prior  to  time  t,  then: 

F (t)  =  1  -  R(t) 

is  the  probability  of  one  or  more  failures  in  time  t, 
where:  R(t)  =  e->,t 

is  the  probability  of  operating  successfully  for  a  time,  t. 

The  Exponential  distribution  is  characterized  by  a  constant 
failure  rate  and  MTBF  (=  1/X). 

The  distribution  is  valuable  if  properly  used.  It  has  the  advan¬ 
tage  of: 

1.  Single  easily  estimated  parameter. 

2.  Mathematically  easy  to  work  with. 

3.  Applicable  fairly  widely. 

4.  Is  additive  -  that  is,  the  sum  of  a  number  of  independent 
exponentially  distributed  variables  is  exponentially  distrib-  • 
uted. 

Care  must  be  taken  to  insure  its  limitations  are  not  exceeded. 

It  arises  from  a  Poisson  process  and  is  applicable  only  where 
such  a  process  exists.  Its  parameter  is  the  mean  failure  rate. 

The  reciprocal  is  the  mean  time  between  failures  (MTBF)  and  is 
often  used  as  a  reliability  goal.  Care  must  be  taken  in  inter¬ 
pretation  if  this  is  done.  If  a  certain  model  of  equipment  has 
an  exponential  distribution  of  life  lengths  and  achieves  a  MTBF 
of  500  hours,  this  does  not  mean  that  most  equipments  of  this 
model  will  run  about  500  hours  before  failing.  In  fact,  63  per¬ 
cent  will  fail  prior  to  this  time,  since  the  probability  of  success 
is  defined  as  R  =  e"^  where  Xt  equals  1  (that  is  t  *  MTBF  and 
Xt.  =  pjn-4F  =1).  The  value  of  e~l  is  0.368,  or  the  reliability 
of  the  unit  to  time  =  MTBF  is  about  37%.  Figure  9-8  displays 
the  Exponential  ‘Functions. 

1.3  POISSON  DISTRIBUTION 


The  Poisson  distribution  arises  from  a  very  large  number  of  trials 
each  with  a  very  small  probability  of  occurrence.  The  distrib¬ 
ution  is  discrete,  referring  to  failures  per  numbers  of  trials 


EXPONENTIAL  DISTRIBUTION 
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rather  than  time  to  failure.  As  in  the  case  of  the  binomial,  the 
sum  of  probabilities  equals  unity.  The  Poisson  failure  density 
function: 


f  (r) 


where 


r  =  number  of  failures 


n  “  number  of  trials 

p  =  probability  of  failure  on  any  one  trial. 

We  use  p  as  the  probability  of  failure  in  the  Poisson  distrib¬ 
ution  (instead  of  probability  of  success  as  in  the  binomial) 
because  we  wish  to  use  the  distribution  in  reliability  areas  in 
which  the  probability  of  failure  is  a  small  number.  The  product 
np  should  be  relatively  constant. 


xhe  probability  of  r  or  less  failures  in  n  trials 


F(r) 


r 

■i 


1  e~np 
il 


The  reliability  R  «  1  -  F(r) 


We  can  think:  of  a  test  period  of  10  hours  being  broken  up  into 
milli-seconds.  Each  milli-second  is  an  independent  trial  (assum¬ 
ing  instantaneous  repair  to  any  failures  that  occur) .  There  are 
r  **  36  x  10®  trials,  each  with  a  small  probability  of  :  failure. 

If  the  failure  rate  of  the  equipment  is  X  *  .001  failures  per 
hour,  the  probability  of  failure  on  one  (milli-second)  trial 

p  =  .001  x  ■■■  1  "/ji —  The  product  np  »  .01  \ 

36  x  lCr 


The  distribution  is  also  useful  in  the  consideration  of  a  large 
population  of  parts,  each  with  a  small  probability  of  failure. 

Xf  the  product  np  is  constant  and  is  the  expected  ntnber  of 
failures  on  a  single  trial  (Which  in  the  case  of  the  exponential 
distribution  is  Xt)  then  the  function  ?(r)  gives  the  probability 
of  having  r  or  more  failures  on  the  trial.  The  probability  of 
having  one  or  more  failures  on  a  given  trial  is  the  exponential 
distribution  function 
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F  (t) 


Xt  e 


-Xt 


and  the  probability  of  success  on  the  trial  is 


R(t) 


figure  9-11  displays  the  Poisson  function. 

1.4  NORMAL  DISTRIBUTION 

The  normal  failure  density  function  is 


f(x)  =  — — 
•Jive* 


-(x-n)a/a<r3_ 


n(u,  c) 


where  tr3  is  the  variance ,  u  is  the  MTBF  and  x  is  the  observed 
time  to  failure. 


The  cumulative  distribution  function  iss 


F(x) 


e-(x-H>a 


dx  =  1  -  R(x) 


The  reliability  distribution  is,  of  course: 

R(x)  =  1  -  F(x) 

The  Normal  is  useful  in  reliability  mathematics  for  two  reasons : 

1.  Wide  applicability  due  to  the  central  limit  theorem. 

2.  Provides  a  direct  description  of  the  distribution  of 
times  to  failure  under  certain  conditions. 

The  distribution  is  continuous  and  is  a  two  parameter  distrib¬ 
ution.  The  mean  and  variance  are  particularly  meaningful,  since 
it  is  a  syromatric  distribution.  The  major  direct  area  of  relia¬ 
bility  application  is  in  describing  the  distribution  of  wearout 
failures.  Considerable  empirical  evidence  has  shown  that  in 
purely  mechanical  assemblies  that  this  is  a  good  approximation. 

The  distribution  is,  however,  easy  to  work  with  and  has  several 
valuable  properties.  The  simple  transformation. 
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a 

transforms  at-y  Normal  variable  x  to  the  so  called  standardized 
Normal  variable  z  which  has  a  zero  mean  and  unit  variance. 

The  frequency  and  the  cumulative  distributions  of  the  standard 
Normal  variable  are  tabled  in  nearly  every  statistical  text. 

The  Normal  distribution  also  has  the  additive  property.  Figure 
9-13  summarizes  the  characteristics  of  the  Normal  distribution. 

1.5  LOGARITHMIC  NORMAL  DISTRIBUTION 

The  three  parameter  log  normal  frequency  function  is  s 


f(t)  = 


a  (t-uj) 


J2rt 


exp 


r 


-(ln(x~u>)-ua 

- 2? - 


•] 


t  >  «J 
0)  i  O  i  t 

where  p,  is  the  mean  of  log  t 

«ra  is  the  variance  of  log  t 
u>  is  the  location  (threshold)  parameter 
The  cumulative  function  is  then 

-  Wii  f  £  exp  [r.Un ax 
o 

R(t)  =  1  -  F(t)  4 - - 

The  log-normal  distribution  is  a  transformation  of  the  Normal 
distribution  in  which  logarithm  of  the  time  is  used  as  the 
variable,  instead  of  the  time.  A  log  normal  distribution  plotted 
on  semi- logarithmic  paper  would  appear  a  normal  curve. 

It  may  be  shown  that  the  Log  Normal  distribution  applies  to 
situations  in  which  several  independent  factors  all  exert  an 
influence  on  the  final  outcome  of  a  given  event  not  in  a  simple 
additive  fashion  but  rather  according  to  (1)  the  magnitude  of  the 
factor  and  (2)  the  importance  of  the  event  at  the  time  in  Which 
the  particular  factor  is  applied.  The  distribution  has  been 


NORMAL  DISTRIBUTION  FUNCTION 
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found  to  be  ive  of  many  cases  of  distributions  f 

times  to  repair.  It  is  used  in  prediction  and  demonstration  of 
mean  times  to  i.  epait  in  MIL-M-2.3  3  i  J  for  example.  ( Figure  9-15 
show;  this  distributi  n)  . 

1 . 6  GAMMA  D ISTR IB17T FTT 


The  Gamma  failure  density  function  can  expressed  as 

t 

B 


f(t) 


1 


a  1  B 


a+  i 


f 


at  e 


t  2  0 

fl  >  0 


■y  >  -1 


where  t  is  time  to  failure 

o-  is  the  shape  parameter 
3  is  the  scale  parameter 


The  failure  rate  is  constant,  increasing  and  decreasing  accord¬ 
ingly  as 


n  =  0,  a  >  0,  j  <  0. 

The  cum u ltd  ive  density  function  is  not  ex  nr ess nb 1 e  in  elementary 
mathernat  i  cal  terms  except  when  a  is  a  posit  ive  intep'r,  but  may 
be  found  tabula*  ed  in  tables  of  incomplete  Gr.mrna  tunct  ions. 

It  has  the  form: 


m 

F  ( t )  ; 

w 
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3  w 

a  1  B 
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n  trials.  The  Gamma  distribution  is  that  of  times  to  failure 


.mu  is 


continuous  over  the  time  axis. 
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The  Gamma  distribution  will  describe  varying  failure  modes  or  ‘jp. 

combinations  thereof.  The  distribution  will  become  Exponential 
when  heterogeneity  exists  among  the  failure  modes.  As  hetero¬ 
geneity  decreases,  the  distribution  approaches  the  Normal. 

Figure  9-17  displays  the  variation  in  the  frequency  for  varying 
parameter  values. 

1.7  WEI BULL  DISTRIBUTION 

The  Weibull  distribution  is  characterized  by  the  failure  density 
function. 

f(x)  =  |  (x-u))*"1  exp 


for 

u>  *  x  <■  • 
a  >  0 
P  >  0 


ocooo  ooo  o  © 

©  <n  (JO  t>  SC  ic  ■q*  CVJ  SN  -« 


aamiv^i  jo  adn  .siOshj 


of  the  frequency  function. 


u)  is  the  location  parameter,,  and  determines  the  point  of  origin 
of  the  curve, 

a  is  the  scale  parameter.  The  real  positive  Pth  root  of  <y  deter¬ 
mines  the  dispersion  of  the  frequency  function  about  the  mean. 

The  frequency  function  of  several  contributing  failure  modes  can 
be  represented  as  a  composite  Weibull  frequency  function  such  as: 


F(x) 


Bi  B»-l 

a~  (x  -  ®x) 


J  +  2aa 


Pa-1 


exp 


[ 


-(x-COfl  )Ba 


] 


This 


can  be  approximated  by  the  function 

fix)  =  §£7  x  Sl_1  exp[^-]  for  o  *  X  <  a 

▼ 

=  1^-  (x-o))®3-1  exp^~  ^ X~(”~ •  f°r  «  <  X  <  «*». 


As  can  be  seen,  the  Weibull  is  essentially  a  three  parameter 
although  u>,  the  location  parameter  (more  appropriately,  the  delay 
parameter  in  reliability  studies)  is  sometimes  set  equal  to  zero, 
and,  thus,  only  two  parameters  remain  to  be  estimated.  This  is 
not  generally  advisable,  however,  and  the  real  flexibility  of  the 
Weibull  distribution  arises  from  the  judicious  use  of  all  three 
parameters.  The  0  parameter  is  the  most  powerful  of  the  three, 
in  a  sense,  since  it  controls  the  general  shape  of  the  curve  and, 
thus,  is  the  variable  which  permits  that  such  a  wide  variety  of 
curve  types  are  included  within  the  Weibull  family.  The  or  is  a 
scale  parameter  which  plays  essentially  the  same  role  as  n  in  the 
Normal  Distribution.  The  characteristics  of  Weibull  distribution 
are  described  in  Figure  9-19.  Special  paper  developed  for  graphi¬ 
cal  solution  to  the  Weibull  parameters  is  available. 

2.  ACQUISITION  OF  DATA 

2 . 1  NATURE  OF  FAILURES 


The  prediction  or  assessment  of  reliability  is  actually  an  eval- 
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uat  5  on  of  unreliabili  ty  It  is  the  rate  at  which  failures  occur 
that  we  use  for  the  measure  of  unreliability.  The  nature  and 
underlying  cause  of  failures  must  be  identified  and  corrected  to 
improve  reliability.  Reliability  data  consist  of  reports  of 
failures  and  reports  of  duration  of  successful  operation  of  the 
monitored  equipment. 

2.2  USES  OF  RELIABILITY  DATA 

Reliability  data  is  used  for  three  main  purposes: 

(a)  To  verify  that  the  equipment  is  meeting  its  reliability 
requirements . 

(b)  To  discover  deficiencies  in  fhe  fpment  to  provide 
bases  for  corrective  action. 

(c)  To  establish  failure  histories  for  comparison  and  for  use 
in  prediction. 

Reliability  data  can  also  be  useful  in  providing  information 
abwUt  logistics,  maintenance,  and  operations.  The  data  can  provide 
a  good  estimate  of  spare  parts  requirements.  With  respect  to 
maintenance,  reliability  data  make  it  possible  to  estimate  the 
degradation  and  wear-out  characteristics  of  parts  and  components. 
From  this  information,  not  only  can  effective  preventive  mainten¬ 
ance  routines  to  control  frequent  trouble  areas  be  developed,  hut 
also  an  estimate  can  be  obtained  of  the  number  of  maintenance  man¬ 
hours  required  to  assure  a  desired  level  of  reliability. 

2.3  VALIDITY  OF  DATA 


It  is  important  that  the  data  be  factual  so  that  a  high  degree  of 
credence  may  be  placed  in  the  conclusions  derived  from  it.  In¬ 
complete  and  inaccurate  reporting  will  inevitably  lead  to  either 
complete  loss  of  confidence  in  the  data  or  to  incorrect  conclus¬ 
ions  and  hence  incorrect  decisions  and  actions  based  on  the  con¬ 
clusions  . 

To  assure  that  the  information  is  valid  requires  that  the  methods 
and  procedures  applicable  to  the  collection  of  the  data  be  clearly 
defined.  The  personnel  responsible  for  the  data  collection  should 
be  carefully  selected  and  adequately  trained  not  only  in  the 
methods  and  procedures  of  reporting,  but  also  in  the  analysis  of 
the  data  and  the  uses  to  which  it  will  be  put  to  enable  intelli¬ 
gent  and  responsive  reporting. 
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£.4__  FACTORS  OF  IMPORTANCE  IN  RETORTING 
Reports  of  failures  should  contain,  as  a  mi  nimum : 


(a)  Time  and  d,a*-e  failure  occurred. 

(b)  Location  (Ship  or  Station) . 

(c)  Identification  of  failed  part  or  assembly  by  name,  stock 
number  and  serial  number  where  appropriate. 

(d)  Identification  of  higher  level  of  assembly  in  which  part 
or  assembly  failed  by  name,  stock  number  or  designation  (as 
AN/SRR-13 )  and  serial  number. 

(e)  Symptoms  or  nature  of  the  failure. 

(if)  Cause  of  failure,  including  causes  such  s  operator  error, 
result  of  another  part  failure  (secondary  failure),  use  beyond 
normal  life  expectancy,  extreme  environment  (temperature,  shock, 
etc . )  . 

(g)  Duration  of  operating  time  since  last  failure  (computed 
from  operating  log). 

(h)  Circumstances  surround ino  the  failure,  with  particular 
reference  to  any  abnormalities  noticed. 

Entries  in  the  equipment  operating  log  will,  be  used  to  establish 
or  verify  mean  life  (meantime  between  failures)  of  the  equipment 
reported  on.  It  should  also  be  used  to  verify  the  use  of  the 
distribution  of  times  to  failure  assumed  in  predictions  or  assess¬ 
ment  of  reliability  parameters .  As  was  discussed  in  Chapter  5, 
the  exponential  distribution  of  times  to  failure  is  normally 
assumed.  Data  indicating  that  certain  parts  or  assemblies  follow 
a  Weibull  or  normal  distribution  with  a  typical  wear-out  or  end 
of  life  characteristic  will  permit  the  establishment  of  realistic 
replacement  schedules,  with  consequent  improvement:  of  reliability. 
Equipment  operating  logs  should  include,  in  addition  to  the 
identification  data  (ship  or  station,  period  covered,  equipment 
designator,  manufacturer,  etc.): 

(a)  Recorded  operating  time  prior  to  the  start  of  the  period 
(e.g.,  reading  of  installed  time  recording  device). 

(b)  Duration  of  each  operation  of  th<  equipment  during  the 
period,  with  purpose  for  which  operated,  abnormal  environments 
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and  reason  for  discontinuing  operation  (such  as  failure,  end 
of  exercise,  test  completed,  etc.) 

(c)  Length  of  time  and  manhour o  expended  in  maintenance 
(preventive  or  corrective),  including  reasons  for  maintenance, 
if  no^  otherwise  clearly  indicated. 

Field  or  use  data  acquired  through  failure  reports  and  operating 
logs,  while  useful,  cannot  be  termed  conclusive.  The  data 
collected  is  usually  incomplete,  inconsistent,  and  inaccurate. 

The  influence  cf  intangibles,  such  as  the  variation  in  capability 
of  operating  personnel,  or  procedures  used  in  maintenance  and 
trouble-shooting ,  and  the  hesitancy  of  some  personnel  to  admit 
errors  when  they  can  be  hidden,  confuse  the  data  and  make  intelli¬ 
gent  interpretation  of  the  data  difficult. 

2.5  METHODS  OF  GATHERING  RELIABT LITY  DATA  --  FIELD  DATA 

There  are  two  usual  methods  of  gathering  data:  (a)  at  random 
from  various  installations  or  (b)  on  the  basis  of  controlled 
programs  using  data  from  units  functioning  under  operational 
conditions  in  accordance  with  a  fixed  routine. 

Experience  has  shown  that  it  .is  usuallv  more  advantageous  to 
gather  data  by  sampling  techniques,  be  :ause  they  are  more  rapid 
and  less  costly  to  the  customer.  Howi  /er ,  sampling  experiments 
cannot  always  be  arranged,  and  difficulty  has  been  encountered 
even  when  the  government  is  the  customer.  Other  methods ,  such 
as  simulating  field  conditions  in  a  laboratory  and  calculating 
the  resultant  reliability,  or  determining  the  failure  rates  of 
parts  or  components  under  simulated  conditions  of  operation  and 
calculating  the  system  reliability,  can  be  used.  Simulated 
experiments,  if  performed  properly,  can  provide  an  estimate  of 
operational  reliability,  but  it  must  be  borne  in  mind  that  the 
methods  used  for  simulating  actual  field  conditions  are  at  best 
a  guess.  Statistically  designed  experiments  should  be  develop*’  ’ 
with  the  assistance  of  a  completent  statistician  to  improve  the 
efficiency  of  the  experiment.  There  are  no  good  substitutes  for 
the  actual  thing.  Nevertheless,  in  some  instances,  the  results 
obtained  by  simulation  methods  have  often  been  nroven  to  be  more 
realistic  than  those  based  on  actual  field  data  gathered  through 
uncontrolled  programs.  Field  failure  data  obtained  through  a 
controlled  program  would,  however,  be  superior  to  laboratory 
data  because  the  former  reflect  actual  operitional  conditions. 

The  factors  to  be  considered  in  choosing  the  tvpe  of  program  are 
(a)  the  ’f'gree  of  assurance  required  that  the  data 


obtained  reflect  an  accurate  picture  of  equipment  reliability, 

(b)  ti  e  amount,  of  data  required,  (cl  the  period  of  time  over 
which  the  data  must  be  accumulated,  t.'’d  (d)  the  relative  costs  of 
various  programs .  When  time  permits,  a  controlled  field  program 
or  a  field  and  laboratory  combination  program  should  be  used. 

2.6  BUSH  It'S  DATA  SVSTEMS 

2.6.1  Electronic  Failure  Reporting  System:  Since  August  1961, 
all  active  ships  and  shore  stations  furnish  *he  Bureau  with  two 
reports  ( 1 ) : 

(a)  NAVSHIP^  *855,  Electronic  Equipment  Operating  Log; 

(b)  DD787  (PROPOSED)  (Report  BuShios  10550-1)  Electronic 

Equipment  Fai lure/Rcplacement  Report, 

Equipment  failures  are  due  to  failures  of  integral  parts,  units 
or  plug-in  assemblies.  The  combined  equipment/part  failure  re¬ 
port  concept  allows  for  the  determination  of  accurate  data 
essential  to  reliability,  maintainability  and  availability  figures 
of  merit.  It  should  be  emphasized  however,  that  no  useful  part 
data  is  sacrificed  t^.  obtain  these  figures  of  merit.  An  important 
feature  of  this  program  is  the  limiting  of  reports  only  to  select¬ 
ed  priority  equipments.  Reporting  requirements  have  been  elimin¬ 
ated  for  obsolete  and  obsolescent  equipment.  This  in  itself  im¬ 
proves  the  paper  work  loan  as  compare'’  to  the  previous  requirements 
of  reporting  on  all  equipments. 

The  two  forms  have  been  designed  to  provide  the  basic  data  nec¬ 
essary  for  the  accurate  calculation  of  reliability  and  maintain¬ 
ability  figures  of  merit,  s  h  as  the  following;  (a)  Mean-Time- 
Between-Failures ,  (b)  Mean-Time-To-Repair ,  (c)  Down-Time,  (d) 

Availability,  (e)  Failure  rates  and  ( f)  Replacement  (consumption) 
rates.  In  addition,  the  Failure/Replacement  Report  form  provides 
the  necessary  identification  data  and  conditions  of  failure  in¬ 
formation  necessary  for  comprehensive  engineering  analyses  of 
high  failure  rate  items.  The  majority  of  Failure/Replacement 
and  operational  time  data  to  be  processed  and  analyzed  in  the 
BUSHIPS  Failure  Reporting/Analysis  Program  will  be  initiated  and 
submitted  by  Navy  Technicians.  The  success  or  failure  of  this 
program  will  therefore,  be  dependent  upon  the  extent  to  which 
the  technicans  are  motivated  to  provide  as  complete  and  accurate 
data  as  is  possible.  In  the  development  of  the  new  reporting 
forma  and  instructions,  all  aspects  were  considered  from  the 
technician's  point  of  view  to  determine  those  factors  that  would 
assist  in  motivating  him  to  provide  the  quality  of  reporting 
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essential  to  the  program's  success. 

Provisions  have  been  incorporated  into  the  new  Failure/Replacement 
Report  form  to  allow  for: 

1.  Reporting  plug-in  assembly  failures  and  repairs. 

2.  Reporting  of  failures  or  replacements  in  units  which  are 
auxiliary  to,  but  not  part  of,  equipment. 

3.  Reporting  failures  or  replacements  identified  by  either 
the  unit  or  block  numbering  system  of  reference  designations, 
(se.  MIL-STD-16). 

4.  Better  identification  of  primary  part  failures  (the  parti¬ 
cular  part  of  a  cluster  of  part  failures,  primarily  responsi¬ 
ble  for  an  equipment  failure,  i.e.,  a  shorted  capacitor  (prim¬ 
ary)  ..hi  :  t  v.  m.  =od  burned  out  resistors  (secondary)). 

5.  Determination  of  reason  for  excessive  downtime,  i.e., 
awaiting  parts  not  locally  available,  repair  beyond  ship  or 
station  capabilities,  unfavorable  weather  or  sea  (working) 
conditions,  etc. 

6.  Segregating  operational  failures,  preventive  maintenance 
(POMSEE)  replacements,  unscheduled  maintenance  rep! moments , 
stock  defective  items,  and  repairs  made  to  replaceable  units 
or  plug-in  assemblies. 

r.  sound  ana  lysis  system  was  planned  in  a  manner  which  will  point 
out  those  parts  which  are  failing  or  being  replaced  at  a  much 
higher  rate  than  others  within  its  particular  group.  Since  it  is 
not  feasible  to  compre  the  failure  rate  of  a  beam  power  klystron 
with  that  of  a  low  voltage  rectifier,  categorization  of  parts 
into  homogeneous  groups  was  accot.pl  ished . 

Prior  to  conducting  an  engineering  analysis,  various  operational 
parameters  will  be  investigated.  Problems  resulting  from  an  in¬ 
dividual  equipment,  ship  or  even  equipments  instal'od  in  the  same 
class  of  ships  may  well  be  the  result  or  improper  installation, 
poor  maintenance  practices,  etc. ,  and  therefore,  in  these  cases, 
engineering  analysis  associated  with  circuitry  should  be  avoided. 
The  possibility  of  concurrent  random  replacements  on  a  fleet-wide 
basis  cannot  be  overlooked.  Replacement  cost,  amount  of  actual 
down-time,  and  maintenance  time  resulting  from  the  failure  oi 
replacement  will  likewise  be  considered. 
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rnly  after  all  these  conditions  have  been  investigated  will  a 
decision  be  made  as  to  whether  or  not  to  conduct  an  engineering 
analysis-  The  engineering  analysis  encompasses  factors  such  as 
investigations  of  circuit  design  and  application,  replacement 
characteristics,  associated  replacement,  all  reported  trouble 
information  (e.g.,  cause  and  type  of  failure),  physical  and 
environmental  factors,  rr  ■  cut  equipment  modification,  etc. 

The  development  and  implementation  of  a  program  incorporating 
suitable  reporting  forms,  systematic  data  preparation  techniques, 
sophisticated  data  processing  programs,  and  sound  engineering 
analysis  criteria  is  of  little  value  without  feedback  of  in¬ 
formation  to  interested  agencies.  This  program  has  been  developed 
so  that  pertinent  information  will  be  distributed  to  the  bureau 
of  Ships,  major  equipment  contractors,  and  other  agencies,  as  well 
as  tc  the  technician  originating  the  reports.  Specific  schedules 
have  been  established  for  each  periodic  report. 

A  report  has  been  designed  specifically  for  distribution  to  con¬ 
tractors  a<_  direction  of  L'..u  Sait-au  i  oiiips.  The  report 

will  provide  a  complete  time- frame  history  of  the  contractor's 
equipment.  The  following  figures  of  merit  will  be  provided, 
based  on  failures  during  operation:  the  mean-t ime-between- failures , 
the  average  repair  time  per  failure. 

The  contractor  will  also  be  provided  with  individual  summations 
bv  Equipment  Model  Designation  in  operating  condition  and  ship  or 
station  code  for  the  following: 

1.  Total  failures  (total  number  or  "Operational  Failure" 

Reports . ) 

2.  Total  operating  time. 

3.  Total  repair  time. 

The  association  between  failure  or  replacement  of  parts  and  part 
provisioning  is  obviou-.  Since  there  is  a  definite  need  for 
guidance  when  provisioning  for  new  equipments  arid  establishing 
reprovisioning  policies  for  old  equipments,  the  application  of 
fleet  failure  or  replacement  data  toward  this  goal  seemed  desir¬ 
able.  Failure  rates  listed  by  Federal  Stock  Number  and  based  on 
realistic  population  and  operating  time  figures  should  provide 
firm  guidelines  for  establishing  spare  part  requirements.  The 
combination  of  failure  rates  and  average  equipment  operating  time 
form  a  bas  13  for  reprovisioning. 
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The  effectiveness  of  an  established  spare  parts 
for  new  equipment?  can  be  determined  if  data  is 
Steps  can  be  taken  to  provide  additional  spares 
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Plans  have  been  developed  to  provide  BUSH IPS  and  the  Electronics 
Supply  off i  'O  with  the  following  information: 

1.  Failure  rites  by  Federal  Stock  Nun, her. 

2.  Average  monthly  hours  of  operation  .  f  equipment  models . 

3.  Replacements  of  Federal  Stock  ‘-lumbers  in  new  equipments. 

4.  Failure/ Replacement  information  on  major  units  and  elec¬ 
tronic  assemblies  of  modular  construct  ion. 

5 .  Maintenance  information  for  major  units  and  electronic 

assemblies  of  modular  construction. 

The  majority  of  this  information  is  directly  related  to  the 
Federal  Stock  Number  since  the  supply  system  uses  this  form  of 
nomenclature.  To  provide  the  information  just  mentioned,  it  »as 
necec  -  ary  to  develop  specific  data  processing  programs  and  pro¬ 
cedures  which  orientate  and  present  failure  replacement  data  ir. 
a  format  suitable  for  part  provisioning  and  reprov is ioni  n.: 
appl icat ion. 

2.6.2  Maintenance  Data  Collection  System:  The  Maintenance  oat 
Collect  ion  (M.po-  System  is  designed  to  provide  all  level.-  of 
management  ' n  the  Navy  Department  with  essential  data  that  can 
be  summarized  by  moans  o  f  data  processing  equipment  into  useful 
management  reports.  At  the  present  time,  the  Navy  does  not  have 
a  method  for  collect  ir.  t  maintenance  information  in  a  usable  for¬ 
mat  .  Through  the  use  of  coded  entries,  on  standard  forms ,  all 
equipment  maintenance  performed  in  the  Navy  will  be  collected. 

All  maintenance  performed  will  be  recorded  on  the  prescribe f 
forms  -  OPNAV  Form  4'’ 00-2  and  4" 00-2 a  -  by  the  person  performing 
the  maintenance.  The  codes  to  bo  used  will  be  listed  in  the 
pert  inent  Equipment  Identification  Code  ( E I C )  Manual.  Trier  e  art- 
four  separate  manuals;  Operations,  Weapons,  Engineering,  and  Hul 
and  Miscellaneous.  The  blocks  on  the  forms  are  identified  by 
numbers  and  letters.  The  coded  entries  will  be  punched  into  the 
dat*  cards. 
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The  following  is  a  recommended  method  of  incorporating  MDC  within 
the  ship's  organization.  An  officer,  preferably  in  the  engineer¬ 
ing  department,  should  be  designated  the  Maintenance  Data  Collect¬ 
ion  Officer  as  a  collateral  duty.  He,  in  turn,  should  have  a 
petty  officer  assistant  whose  primary  duty  will  be  MDC  Petty 
Officer.  An  office  such  as  the  Engineering  Log  Room  should  be 
designated  the  MDC  center  for  collection,  review,  mailing,  dis¬ 
tribution,  and  filing  of  the  MDC  forms  and  reports.  The  leading 
petty  officer  of  each  rate  or  space  will  be  the  Maintenance  Group 
Supervisor.  He  will  review  each  document  submitted  by  his  sub¬ 
ordinates.  He  is  responsible  for  ensuring  that  all  MDC  forms 
are  complete  and  accurate.  Incomplete  or  inaccurate  maintenance 
data  will  result  in  erroneous  manning,  equipment  budgeting,-  or 
work  requirements  action  at  higher  levels  of  command. 

The  following  data  is  entered  on  the  standard  form: 

(a)  Ship's  name  and  hull  number. 

(b)  Equipment  identification  code  from  El'C  Manual  for  part 
repaired. 

(c)  Classification  of  the  group  that  performed  the  maintenance 
by  code  number  from  EIC  Manual. 

(d)  TyPe  of  maintenance  action,  (failure  mode,  frequency  of 
routine  maintenance,  alteration  or  manufacture  of  new  item). 

(e)  Manhours  expended.  ' 

(f)  Date  action  taken 

(g)  Serial  number  of  equipment. 

(h)  Written  description  of  malfunction  or  reason  for  main¬ 
tenance  action. 

(i)  Written  description  of  corrective  or  maintenance  action. 

(j)  List  of  spare  parts  used  (CID,  APL,  or  AN  Numbers). 

2.5.3  Operations  Reporting  System:  The  OR  information  such  as 
obtained  in  F3M  programs  indicates  the  types  and  distributions 
of  problems  being  experienced,  the  corrective  action,  the  process 
by  which  it  is  applied  and  the  effectiveness  of  the  corrections. 
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PROBLEMS  WITH  EXPERIENCE  DATA 


Some  of  the  reasons  traditionally  given  for  the  limited  use  of 
field  and  operational  data  are: 

1.  Problem:-,  in  Data  Retrieval  (what  data,  can't  find  it, 
too  cumbersome,  takes  too  long,  costs  too  much,  etc,). 

2.  Problems  with  Data  Accuracy  (can't  believe  what  its  say¬ 
ing,  technicans  do  not  record  properly,  most  failures  result 
from  outside  of  its  control  and  therefore  should  not  get 
blamed,  it  never  really  tells  me  why  it  failed,  etc.). 

3.  Problem  in  Understanding  (how  to  use  it,  what  does  it 
mean,  hou  does  one  relate  the  problem  on  hand,  etc.). 

3.1  TRADITIONAL  CONCEPTS 

Possibly  the  most  common  and  certainly  the  most  destructive  con¬ 
cept  concerning  the  retrieval  of  experience  data  is  that  it  cun 
be  retireved  in  the  exact  orientation  and  form  required  by  the 
"user".  The  probability  that  this  can  be  done  is  at  least  as 
low  as  that  of  obtaining  computer  solutions  to  problems  in  res¬ 
ponse  to  interrogations  express*  dii  ^ctly  in  the  form  of  applied 
engineering  equation.  The  analogy  is  hignly  appropriate.  One 
cannot  retrieve  information  from  the  computer  without  first  con¬ 
verting  the  engineering  language  to  computer  language,  and  then 
programmming  it  in  a  manner  dictated  vy  both  the  type  of  in¬ 
formation  stored  and  the  computation  capability  of  the  computer. 
The  seme  is  true  for  the  stored  experience  data.  The  researcher 
must  understand  the  type  of  data  stored  and  how  to  retrieve  it. 

All  too  frequently  the  reliability  engineer  has  rejected  much  of 
the  current  existing  scurce  of  knowledge  in  his  search  for  data 
which  explicitly  displays  the  parameters  of  his  favorite  equation, 
R  =  Thus,  he  be]  i eves  that  he  must  find  a  single  data 

format  which  directly  produces  values  for  MTBF.  The  impact  of 
this  can  be  estimated  by  noting  that  the  regularity  with  which  he 
insists  that  clocks  be  designed  into  hundreds  of  components 
labeled  "Critical".  At  the  same  time  it  has  led  him  to  reject 
-  i  of  the  existing  experience  data  as  practically  worthless. 
Until  he  has  been  helped  to  understand  that  it  is  not  necessary 
for  tine  to  be  recorded  and  expHcitly  displayed  against  each 
event,  he  will  make  little  progress  in  recognizing  what  infor¬ 
mation  is  ot  potential  value  to  him.  As  soon  as  he  completely 
understand  tint  essentially  all  significant  events  i ->  the  "real" 
world  are  performed  in  time  oriented  sequences,  his  horizons  will 
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be  enormously  expanded.  It  is  then  that  he  realizes  that  almost 
any  component  can  be  related  to  the  time/ sequences  in  which  it  was 
operated.  For  instance,  we  have  two  hydraulic  systems  in  which 
there  are  two  pumps  each  (four  identical  pumps ) .  Any  one  pump 
can  handle  both  hydraulic  systems,  if  necessary.  However,  gen¬ 
erally’  one  pump  on  each  system  is  utilized  with  the  other  pump 
as  a  back  up.  The  pumps  are  alternated  daily  to  accumulate 
approximately  the  same  number  of  operating  hours.  A  cruise  lasts 
60  days.  Therefore  the  total  hours  accumulated  on  this  type  of 
pump  is  2880  hours  on  a  60  (  ay  cruise  par  vessel. 

3.2  MARSHALLING  YOUR  DATA  SOURCES 

The  first  element  to  attack,  and  possibly  most  important,  is  the 
source  existing  within  one's  own  "house1'.  This  basic  stay-  is 
possibly  the  most  difficult  single  element  of  the  entire  task. 

The  marshalling  task  is  unglamorous  and  down  right  tedious.  The 
task  also  requires  that  a  survey  be  made  of  how  people  are  con¬ 
ducting  their  business  and  how  many  "road  blocks"  are  erected  in 
the  process.  Another  problem  which  traditionally  gets  in  the  way’ 
of  a  successful  study  is  the  researcher’s  desire  to  describe  the 
sources  either  as  he  wants  them  to  be  or  as  his  management  would 
like  them  to  be.  Describing  them  as  they  actually  operate  re¬ 
quires  considerable  objectivity  and  tact. 

Each  data  source  must  be  studied  in  detail  with  respect  to  data 
content,  flow,  storage  and  retrieval.  After  all  sources  have 
beer,  carefully  examined,  it  can  be  seen  that  extensive  and  de¬ 
tailed  study  must  be  made  of  both  the  source  and  reduced  data  to 
realize  the  importance  of  implied  cross  reference. 

3.3  EXPERIENCE  DATA  INTEGRATION 


Once  the  survey  has  been  completed  and  the  sources  analyzed,  a 
second  phase  must  be  undertaken,  that  of  developing  the  techni¬ 
ques  to  exploit  the  information. 

Figure  9.8  points  out  that  experience  data  is  only  one  of  the 
four  major  ingredients  necessary  to  determine  the  life  character¬ 
istics  (\t„)  of  an  eauipnent  item.  Another  necessary  portion  is 
the  "ENGINEERING  INFORMATION"  block  while  the  third  is  the 
accurate  integration  by  the  interpreter (s )  operating  on  the  fourth 
ingredient,  the  appropiate  process  (the  equation). 

3.3.1  "Engineering  Information"  ;  As  with  the  use  of  a  computer 
in  the  solution  of  any  engineering  problem,  the  foundation  of 
success  is  built  by  correctly  identifying  and  stating  the  problem 
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indicated  by  Ms/f,  the  selected  failure  mode{s)  defined  approp¬ 
riate  to  the  problem.  In  the  attempts  at  the  solutions  to  the 
reliability  aspects  of  the  design  problems,  this  step  is  the  most 
consistently  neglected  and/or  troublesome.  The  parameters  are 
the  success/' fai lure  criteria  and  the  analogous  experience  data 
deemed  appropriate  to  the  problem.  The  Operations  Problem 
Reports  may  assist  in  the  selection  by  indicating  failure 

mode  and  effect  documented  for  the  analogous  item. 

In  addition  the  retriever  must  have  the  basic  knowledge  and  under¬ 
standing  of  the  functions  of  the  item  as  related  to  input/output 
requirements,  sequence  of  operation,  checkout  and  procedures  of 
the  system  which  uses  this  item,  etc.  "Exposure  to  failures" 
are  generally  reported  at  the  overall-system  or  major  system 
level.  Item  "exposure  to  failures"  and  environmental  stress 
level  must  therefore  be  estimated  by  the  item  time/sequence 
functioning  as  applied  co  the  system. 

3.3.2  Equation  to  Relate  Elements:  The  next  step  is  to  form 
the  appropriate  equation  (s)  w'hi.ch  correctly  relates  the  para¬ 
meters  of  the  problem  to  the  degress  of  detail  and  in  the  form 
appropriate  to  the  existing  data. 

The  process  by  which  the  ■'arious  parameters  can  be  arithmetically 
combined  is  given  by  the  equation  on  Figure  9-33. 


The  numerator  of  the  equation  represents  the  estimated  item 
failures  in  the  given  calendar  time  period,  t„ ,  for  the  selected 
failure  mode(s)  during  the  selected  application  activities. 
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is  the  total  reported  numbers  of  unschedule  removals  for  the 
selected  mode(s).  However,  the  reported  failure  mode(s)  is  not 
always  the  true  failure  mode  (s).  Km  is  th  e  estimated  ratio  (%) 
of  the  true  mode(s)  to  the  reported  mode(s)  determined  from  the 
Failure  Analysis  Reports. 


The  denominator  of  the  equation  represents  the  estimated  "expos¬ 
ure  to  failure"  from  the  selected  application  activities  during 
the  given  calendar  period,  t, .  Activities  are  generally  report¬ 
ed  at  the  system  level.  From  the  system  level,  the  item  "expos¬ 
ure  to  failure"  may  be  estimated  by: 

Et  x  x  x  T-; 

Et  A. 


EXPERIENCE  DATA  APPLICATION 
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EtB  is  the  number  (s)  of  system  cycles  in  the  given  calendar  time 
period,  tB  ,  for  the  selected  application  activity. 

—  is  the  number (s)  of  item  cycles  per  system  cycle. 

Et 

m 

I  is  the  number (s)  of  item  identified  with  the  system. 

Aa 

is  the  functioning  time  per  item  cycle. 

£c  Nj 

Ex  x  „  x  r~  =  Total  item  cycles  during  tn  . 

8  fct  Aa 

Ic  Ni 

E+.  x  —  x  —  x  T  i  =  Total  item  operating  timeduring  t.  . 

Et  Aa  x 

3.3.3  Integration  of  the  Data;  As  implied  in  Figure  9-31,  the 
interpreter  is  the  primary  integrator  of  the  data.  The  first 
step  is  the  job  of  "programming”.  Programming  (forming  the  data 
retrieval  requests)  for  experience  data  generally  encompass 
several  separate  sources  of  data  as  well  as  the  careful  matching 
of  details  and  format  of  the  data  stored  by  the  various  functions. 
Here  we  must  be  aware  that  virtually  no  one  could  collect  and 
store  data  oriented  to  the  almost  infinite  variety  of  problems 
which  are  asked  of  the  data  source. 

Once  the  retrieval  "program”  is  correctly  formed  and  thrt  data 
becomes  available,  it  must  be  processed  (sorted,  ordered,  com¬ 
bined,  etc.)  for  application  to  the  equation (s)  employed  in 
forming  the  value  (s)  for  the  chosen  p^vamter  (s )  . 

Much  information  duplication  will  be  noted  on  a  component  prob¬ 
lem  report(s);  i.e.,  discrepancy,  when  disco. ered,  part  number, 
etc.,  will  be  indicated  by  “code"  and/or  written  description. 

In  addition,  the  sama  problem  may  be  reported  by  many  sources. 
When  the  duplications  and  redundant  reports  are  utilized  with 
the  problem  occurrance  matching  the  r  -tivity  chart,  the  net 
result  cf  the  data  will  be  highly  accurate. 

"Failure  rate"  or  the  reliability  estimate  is  determined  from  the 
ratio  of  the  problem  curve  to  the  activity  curve.  In  general, 
problems  are  well  documented,  collected,  sorted,  and  relatively 
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easy  to  locate  and  retrieve;  nowever,  the  activities  (exposure  to 
failure,  or  successes),  although  well  documented,  are  not  gener- 
nily  collected  and  therefore  making  the  locating  and  retrieval 
much  more  difficult. 

Crude  as  this  method  may  appear,  its  application  has  been  extremely 
helpful  in  clearing  up  the  usual  difficulties  encountered  in  even 
good  data  collection  systems. 

4.  ESTIMATING  PARAMETERS  FROM  THE  DATA 


Assuming  the  data  sources  provide  a  set  of  operating  time  and 
failure  data,  properly  evaluated  to  screen  out  errors  and  data 
not  pertinent  to  the  problem  the  first  step  is  the  assumption  of 
the  underlying  distribution.  This  assumption  must  be  based  on 
physical  characteristics  of  the  problem.  The  clues  provided  in 
the  section  on  failure  density  functions  will  be  helpful  in  most 
cases . 

If  the  exponential  distribution  has  been  assumed,  the  next  quest¬ 
ion  is  "How  well  does  the  data  support  the  assumption?"  The 
technique  for  evaluating  the  fit  is  termed  the  "goodness-of-fit 
test"  . 

4.1  GRAPHICAL  SOLUTION 


A  graphical  procedure  is  useful  for  the  quick  indication  of  the 
validity  of  the  exponential  assumption  provided  that  the  number 
of  observed  failures  is  relatively  large.  One  procedure  is  to 
plot  the  cumulative  test  or  operating  time  against  the  cumulative 
number  of  failures  as  in  Figure  9-34.  If  as  shown  in  the  example, 
the  failures  occur  uniformly  with  time,  the  assumption  of  the 
exponential  distribution  appears  valid. 

4.2  CHI  SQUARE  GOODNESS  OF  FIT  TEST 


A  more  sensitive  test  to  verify  the  assumption  of  the  failure 
distribution  can  be  performed  analytically.  From  the  assumed 
distribution  function,  predict  the  number  of  failures  that  will 
fall  in  each  of  several  arbitrarily  selected  increments  of  time. 
Fqr  each  such  increment  the  predicted  or  expected  number  of 
failures  will  be  np  where  n  is  the  number  of  equipments 
operating  and  p  is  the  probability  of  failure  during  the  time 
in  question.  If  k  increments  are  selected,  compute  the  k 
values  of  e,  the  expected  number  of  failtires  in  the  ith  increment. 
For  each  increment,  count  the  observed  number  of  failure,  ot  . 

The  summation 


DISTRIBUTION  GRAPHICAL  EVALUATION 


CUMULATIVE  OPERATING  TIME 
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k 

V 

i^l 


(ot  -  et  ) 


is  chi-square 


distributed  for  reasonably  large  values  of  the  e,  (eq  >  5).  The 
number  of  degrees  of  freedom  for  the  \S  distribution  is  k  -  1, 
except  where  the  sample  is  used  to  establish  the  expected  value 
of  MTBF.  In  this  case  the  number  of  degrees  of  freedom  is  k  -  2 , 
one  degree  in  effect  being  used  in  the  selection  of  the  MTBF. 


Example:  Test  data  reports  on  an  electronic  system  give  the 


following  times  to 

failure : 

tt  .hours 

27  .0 

39.0 

61.4 

69.6 

86.3 

96.5 

98.2 

10 1.5 

119.2 

128.6 

144.0 

164.6 

180.0 

180.0 

183.8 

198.2 

206.8 

229.1 

2  59.5 

272.6 

286.4 

312.1 

319.3 

339.0 

415.9 

419.5 

609.8 

729.1 

898.7 

1159.0 

To  test  the  sample  against  an 

MTBF  of  3 GO 

hours , 

increments  of 

time  will  be  selected 

so  as  to 

have  an  expected  value  of  six 

failures  in  each  of 

five  time 

increments . 

Selecting  a  uniform 

probability  of  20%  for  each  increment  and 

t 

the  equation 

R  =  e 

solving 

tor  t. 

R 

t 

e 

o 

(e  -  o)» 

e 

.80 

67 

6 

3 

1.50 

.60 

150.3 

6 

8 

.67 

.40 

275.2 

6 

9 

1  .  50 

.20 

483.0 

6 

6 

0 

.00 

m 

6 

4 

.67 

V  = 


4.33 


"'he  probability  from  a  x*  table  us i nq  four  degrees  of  freedom 
gives  a  value  of  about  65'v.  This  means  that  samples  drawn  from 
the  assumed  population  will  give  you  values  in  excess  of  this 
number  (4.33)  about  one  third  of  the  time.  This  represents  a 
reasonably  good  fit. 
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Using  the  data  coll 

ected  the 

total  opera  t 

ing  time  is 

8335  1 

for  30  failures.  T 

he  mean  time  between  f 

a i lures 

9 

of  •  - 

testing  the  sample 

against  an 

assumption 

of  278  hours 

MTBF 

R 

t 

e 

o 

(e  - 

e 

o 

CD 

• 

62 . 1 

6 

3 

1.50 

.60 

142 . 0 

6 

7 

.  17 

,40 

2  55.0 

6 

8 

.67 

.20 

448 . 0 

6 

8 

.67 

• 

o 

o 

00 

6 

4 

.67 

3.67 


From  an  x3  table  using  (k  -  2)  or  three  degrees  of  freedom  the 
probability  if  found  to  be  7  0''.-.  This  is  a  slight  improvement 
indicating  the  lower  MTBF  should  be  preferred. 

4.3  _ TESTING  DATA  AGAINST  OTHER  i  ■  I  ST P. I 8 UT IONS 

The  x3  test  is  applicable  for  testing  the  data  against  any  dis¬ 
tribution,  using  the  parameters  to  be  tested  in  the  same  way  as 
was  done  in  testing  the  exponential.  Details  can  be  found  in 
Chapter  10  or  in  any  good  text  '.m  statistics. 
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Chapter  10 

STATISTICAL  TECHNIQUES 
1 .  SEQUENTIAL  ANALYSIS 

Sequential  Analysis  is  a  procedure  that  leads  tc  a  statistical 
inference  and  in  which  the  number  of  observations  to  be  made  is 
not  determined  before  the  experiment  is  bequn.  The  procedure 
indicates  when  sufficient  observations  have  been  taken  in  order 
that  a  decision  to  accept  or  reject  a  given  hypothesis  can  be 
made  with  predetermined  producer's  and  consumer's  risks,  denoted 
respectively  by  a  and  6.  On  the  average,  fewer  observations  will 
be  required  by  this  procedure  and  its  use  will  not  increase  the 
value  of  a  and  6. 

1-1  PROCEDURE  OF  TESTING  A  HYPOTHESIS 


Observations  are  taken  one  at  a  time.  After  every  obsei  ation, 
a  decision  is  made  to  accept  the  hypothesis,  to  reject  the  hypo¬ 
thesis,  or  continue  taking  observations.  In  order  to  determine 
■which  of  these  decisions  to  make,  the  critical  region  for  each 
sample  size  must  b>  determined.  To  do  this,  Pom,  the  probability 
that  m  observations  collected  in  the  successive  sampling,  would 
occur  if  the  hypothesis  H0  were  true,  and  P-t^,  the  probability 
that  these  observations  would  occur  if  the  alternative  hypothesis 
were  in  fact  true.  To  compute  Pom  the  assumption  is  made  that 
the  hypothesis  H  is  actually  true.  In  a  similar  manner,  to  com¬ 
pute  P]_m,  the  hypothesis  is  assumed  to  be  true.  When  l\,ni  is 
much  larger  than  Pjm,  H0  is  to  be  accepted.  When  P^T  is  much 
larger  than  Pom ,  is  to  be  accepted.  If  I\,n,  is  apprcximat  oly 
equal  to  Plm-  sampling  will  continue.  A  simple  mathem.at  ical  re¬ 
lationship  that  will  express  these  conditions  is  the  ratio  of  P, 
to  P  .  The  following  inequalities  were  proven  by  Wald  (1)  to 
characterize  the  sequential  test. 


If, 


Lm  1 


accept 


If, 


lm 


am 


,  accept  H'.. 


It, 


1  - 


lm  1  -  r 

w  — —  *-  -  ,  continue  sampling, 

P  a 

cm 


The  above  inequalities  are  illustrated  in  Figure  10-3. 
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Sequential  analysis  is  not  restricted  t-  any  one  type  of  probabil¬ 
ity  distribution,  but  can  be  applied  in  general.  The  arithmetic 
necessary  to  deriving  the  equations  for  the  bounds  of  the  decision 

interval  ..-ill  be  demonstrated  below  us  ina  the  Binomial  Probability 
distribution. 

For  this  case  H0  is  P  =  Pc;  Hx  is  P  =  Px. 

porn  =  p0  -  po  •  9 

d  =  defective 


and , 


g  =  good 

Plm  =  P1  C^1  -  pi) 


then , 


Ira 


P,  (1  -  PJ 

i  i 


Pda 
om  P  ( 1  -  P  ) ^ 
o  o 


Take  natural  logarithms  and  get. 


q  d  1  -  q  9 
!f'  !nr> 

o  o 


p  P 

Ln  (--)  -  d  Ln(-~ ) 

om  o 


g  Ln  ( 


1  -  P, 

_ j 

1  -  P 


Using  the  above  equality  for  Ln(Ppm/Pom)  and  appropriately  con¬ 
verted  toms  of  the  two  basic  inequalities  fcr  the  sequential 
test,  the  following  decision  criteria  may  be  derived: 


.)  . 


Ln  (-JL_) 

1  -  a 


d  Ln 


1 

(■— )  +  9  Ln  (~ 
o 


Decis ion : 


Accept  H 

o 

% 


l 
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2. 


P  1  -  P 

Ln  "  — )  *  d  Ln  (~)  +  g  Ln  (~ - — ) 


-  P 

o  o 


Decision:  Reject  Hq,  i.e.,  accept  R, 


P  1  -  P 

Ln  (~  - )  <  d  Ln  (— )  +  g  Ln  (t - — . 

1  -  a  P  1  -  P 

o  c 


<  Ln  (iil) 


Decision:  Continue  sampling. 

The  above  decision  criteria  can  be  expressed  in  a  slightly  differ¬ 
ent  manner  if  we  note  that  values  of  c ,  a,  P^»  and  PQ  are  to  be 
stated  before  the  test  procedure  is  ue fined. 


Let  Ln  (r-^— )  =  K.  ,  Ln  (— -£)  =  K, , 
l-o  1  a  2 


p  1  -  P 

I  1 

Ln  (~)  =  D  ,  Ln  (1  ----- j  =  C?,  and  g  =  m  -  d. 
o  o 

One  can  restate  the  decision  criteria  as  •''allows: 


1. 

K1  - 

C2m 

;>  d 

(C1 

~  C2) 

2. 

K2  - 

C2m 

*  d 

(C1 

-  <V 

3. 

1 

C2m 

<  d 

(C1 

-  C2>  <  K; 

Accept  H 


Reject  H 


The  decision  boundary  line  functions, 


K, 


C1  -  C2  C1  -  C2 


C2  K2 
m  and 


C1  -  C2  C1  -  C2 


m  , 


are  graphically  presented  in  Figure  10-6. 


f 
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To  illustrate  the  method  of  sequential  testing,  consider  a  test 
of  the  honesty  of  a  coin  that  is  suspected  of  giving  too  many 
heads.  A  Binomial  Sequential  test  will  h"  made  of  the  hypothesis 
HQ :  PQ  ~  0.5  against  an  alternate  hypothesis  =  0.7.  Let 

the  type  I  and  type  II  errors  be  =  .10  ana  6  -  0.20. 


1. 

2. 

3. 


4. 


6. 


7  . 


Kx  =  Ln 
K2  -  Ln 

C  -  Ln 

C0  =  Ln 


(_ — L — ,  =  Ln  (-~)  -  -  1.504 
1  -  v  *9 

(~— )  =  Ln  (-4)  ^  2.07  9 
a  .  1 

P1  7 

(~)  =  Ln  (~)  =  '•■•337 

o 

1  _  P 

frr~r)  =  T,n  {l)  =  °‘5U 

o 

=  0.337  -  (-0.511)  =  0.848 


1.8  +  .6m 


C 


2 


C„ 


m 


2 . 4  +  .6m 


(lower  boundary) 


(upper  boundary) 


The  lower  boundary  line  separates  the  acceptance  region  from  the 
no-decision  region;  it  is,  therefore,  called  the  acceptance  line. 
The  upper  boundary  line  separates  the  rejection  region  rom  the 
region  of  no-decision;  it  is,  therefore,  called  the  rejection 
1  i  ne . 


At  each  toss  of  the  coin,  a  decision  must  be  made  as  to  the 
honesty  of  the  coin.  The  results  of  tossing  a  coin  are  given  in 
Figure  10-8.  From  this  illustration,  it  is  noted  that  the  test 
was  finished  after  13  trials  since  a  decision  was  made  at  that 
time.  The  hypothesis  that  P  =  0.5  was  accepted  over  the  alterna¬ 
tive  F^.  If  the  alternative  had  been  -  0.65  a  larger  number 
of  trials  would  have  been  required,  on  the  average,  to  arrive  at 
a  decision  with  the  same  probabilities  for  the  risks. 

The  result  of  a  sequential  significance  test  analysis  is  a 


Number  of  Trials, 
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nair  of  lines  that  will  divide  the  sample  space  into  three  regions 
such  that  after  each  test  a  decision  can  be  made  to  either  con¬ 
tinue  testing  or  to  accept  one  or  the  other  of  two  hypotheses. 

It  is  then  a  simple  process  to  represent  the  results  of  each  test 
as  a  score  and  to  terminate  the  test  as  soon  as  the  cumulative 
total  of  the  score  has  reached  either  of  two  limits.  In  the 
simpler  cases  the  two  lines  will  be  straight  lines  as  was  the 
case  with  the  coin.  It  will  then  be  necessary  only  to  plot  some 
function  of  the  observations  on  a  chart  where  the  two  straight 
lines  indicate  the  limits  of  interest.  If  these  lines  are  paral¬ 
lel,  they  will  have  the  same  slope.  The  information  needed  to 
plot  them  will  be  the  si  and  their  intercepts.  The  informa¬ 
tion  is  given  below  for  each  distribution  function  discussed. 

The  test  is  then  to  be  terminated  when  one  of  the  lines  is  reached. 
In  practice,  however,  the  line  will  generally  be  crossed;  so  the 
errors  involved  will  actually  be  less  than  that  which  is  speci¬ 
fied. 

It  is  now  desired  to  outline  a  sequential  testing  procedure  for 
several  well-known  distribution  functions.  The  following  defini¬ 
tions  are  necessary: 

'  General  Terms: 

a  =  Producer's  or  Vendor's  risk  -  Probability  of  rejecting 
the  reliability  of  the  system  even  though  the  relia¬ 
bility  is  satisfactory.  -  (The  probability  of  a  type 
I  error)  . 

r  =  Consumer's  risk  -  Probability  of  accepting  the  relia¬ 
bility  of  the  system,  when  the  actual  reliability  is 
not  as  good  as  the  specification.  -  (The  probability 
of  a  type  II  error). 

n  =  Sample  Size  -  number  of  observations. 

n  =  The  Average  sample  number  -  the  expected  number  of 
tests  that  must  be  made  before  a  decision  will  be 
reached. 

m  =  Nunber  of  items  tested. 

hc  =  The  intercept  of  the  line  forming  the  lower  boundary 
line  of  decision. 

■« 

hj  =  The  intercept  of  the  line  forming  the  upper  boundary 
line  of  decision. 
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s  =  The  common  slope  of  the  two  lines  forming  the  boun¬ 
daries  of  the  regions  of  decision. 

Normal  Test  or  "t"  test; 

u  =  Mean  of  normal  distribution. 

=  Standard  deviation  of  a  normal  distribution. 

—  =  The  standard  deviation  of  the  arithmetic  mean  of  the 

■  n 

observations . 

x  =  Mean  of  the  observations  of  x. 

Exponential  test: 

8  -  Mean  Time  to  Failure. 

0  =  Estimate  of  the  Mean  Time  to  Failure. 

e  -  Acceptance  Number  -  A  value  of  time  to  failure  which 
if  exceeded  by  the  sample  mean  time  to  failure  will 
assure  the  required  reliability  coefficient. 

r  =  Number  of  Failures. 

Poisson  test: 

m:.  ,m-  £  Parameter  of  the  Poisson  distribution. 

In  the  general  statistical  testing  situation  the  error  made  when 
a  correct  null  hypothesis  is  rejected  (e.g.,  rejecting  an  item 
meeting  the  specification),  is  called  a  type  I  error  and  the  pro¬ 
bability  of  a  type  I  error  is  called  the  significance  level  -- 
usually  denoted  by  the  symbol  .  The  error  made  when  a  true 
alternative  hypothesis  is  rejected,  (e.g.,  acceptance  of  a  defec¬ 
tive  i t  -m )  is  called  a  type  II  error.  The  probability  of  not 
making  a  type  II  error  is  called  the  power.  Thus,  Power  equals 
one  (1)  minus  the  Consumer's  risk. 

_l_._2 _ NORMAL  DISTRIBUTION 

The  first  distribution  to  be  considered  is  the  normal  distribu¬ 
tion.  A  test  of  the  hypothesis  HQ :  ..  -  against  an  alternative 
H  i  ^  is  developed  for  measurements  from  a  normal  population. 

It  is  important  to  detect  the  difference  between  the  two  means. 
The  type  I  error  >,  is  the  risk  of  asserting  a  difference  when 
none  e<ists,  and  r ,  the  type  II  error,  is  the  risk  of  asserting 
no  difference  when  the  mean  •'  s  really  different. 
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To  construct  the  chart,  convenient  values  are  chosen  for  the  two 
scales.  On  the  horizontal  axis  is  the  number  of  tests  or  obser¬ 
vations  ,  and  on  the  vertical  txis  is  the  function  measured. 

The  sequential  testing  procedure  for  normal  distribution  (with  ■- 
known)  is  described  in  the  following  manner. 

1.  Calculate: 

Slope  of  boundary  lines: 


S  = 


-  i 


Intercepts  of  boundary  lines: 

2  2 

h  -  B  or-  ,  h,  -  A  —  where, 
o  1 


1  -  “o'  A  “  Ln 


,  B  =  Ln 


1  -  rv 


Average  sample  number  required: 


(1  -  h  *  h 

_ o _ i 

-  s 


i  f  -  =  u 


n.  =  - - -  if 

1  ^  -  b 


n 


-h  h 
o  1 


if 


k  +  u. 

o  1 


2.  Plot  the  boundary  lines: 


T  -  h  +  ns 

o  o 


T 1  ~  n  1  *  nS 
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3.  Proceed  with  testing.  A  decision  is  reached  when  a 
boundary  is  crossed. 

1.3  BINOMIAL  DISTRIBUTION 

The  plan  for  sequential  testing  for  the  binomial  distribution 
has  already  been  discussed. 

The  sequential  testing  procedure  for  binomial  distribution  is 
computed  in  the  following  manner. 

1.  Calculate: 

Slope  of  boundary  lines 


L„^- 

qi 

s  = - - - —  ,  where  q 

T  ^  1  T  ° 

Ln  —  +  Ln  — 

P  q 

o  41 


=  1  -  P 


Intercepts  of  boundary  lines 


1  9 

i  -  - — -  where  K,  -  Ln  - 

o  F  q„  1  1  -or 

_  L  Q 

Ln  — -  +  Ln  — 

Po  % 


r  P 1  t  q° 
Ln  - —  +  Ln  — 

Po  qi 


K ,  --  Ln  — 


Average  sample  numbers  required 


v  '"o  1 

P  -  S 


if  P  =  P 


h  ♦  ( 1  -  r  ) ! 


PI  -  S 


if  F  =  P , 
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n 

s 


-h  h 
o  1 


S(1  -  S) 


i  f  p  -  S 


2.  Plot  the  boundary  lines: 


d  =  h  +  mS 
m  o 


d  =  h,  +  mS 

m  1 

3.  Proceed  with  testing.  A  decision  is  reached  when  a  Doundary 
is  crossed. 

1.4  CHI-SQUARE  DISTRIBUTION 


When  the  mean-t ime-to- fai lure  must  be  estimated,  the  Chi-Square 
distribution  is  employed.  It  has  the  following  properties: 

The  frequency  curves  extend  from  zero  to  infinity. 

In  the  case  n  =  1,  the  curve  is  merely  the  positive  half  of 
the  normal  curve;  as  n  tends  to  infinity,  the  distribution 
tends  to  normality,  but  rather  slowly. 

When  n  is  greater  than  1  the  function  is  zero  at  the  origin, 
rises  to  a  mode  at  n  -  2  and  then  falls  off  again  to  infinity. 

The  moments  about  the  mean  are: 


m  -  2n 
m^  =  8n 

2 

m.  =  48r.  +  12n 
4 

The  distribution  function  is  an  incomplete  gamma  function. 

The  sequential  testing  procedure  for  Chi-square  distribution  is 
computed  in  the  following  manner. 

1.  Calculate: 

2 

Values  of  x  /2r  for  upper  and  lower  boundary  lines, 
using  values  of  T ^ ,  T2«  ■>,  and  6  selected  from 
previous  data. 


j 
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2  . 


Plot  the  boundary  lines 


Upper  : 


T 


T 


2r 


Lower:  T  =  T,  — 

1  2r 


3.  Proceed  with  testing.  A  decision  is  reached  when  a 
boundary  is  crossed. 

1.5  POISSON  DISTRIBUTION 


When  an  event  has  only  a  very  small,  constant  probability  of 
occurring,  but  many  trials  are  made  so  that  the  event  does  in 
fact  happen  with  measurable  frequency,  the  number  of  occurrences 
’ s  given  by  the  Poisson  distribution.  While  the  probability  of 
a  given  component  failure  at  any  time  is  very  small,  where  many 
components  are  used  in  a  given  system,  failures  are  distributee 
at  random,  this  fre  juency  will  be  described  iy  the  Poisson  dis- 
tr ibut ion . 

Problems  where  the  Poisson  distribution  applies  are  met  in  con¬ 
nection  with  counting,  for  example,  in  determining  the  number  of 
neutron  particles  reaching  a  counter  in  a  given  time  interval. 

The  sequential  testing  procedure  for  Poisson  distribution  is 
calculated  in  the  f c 1 low  ini  manner. 

1.  Calculate: 


Slop-'  of  boundary  lines 


1 .  v 


Intercepts  of  beun.iarv  l.n< 


Ln  { 


1  -  - 
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Ln  (- - ) 


m 


Ln 


m 


1 


Average  sample  numbers  required  when 
m  =  s 


n  n„  and 


n 

o 


(1  -  .  ) h  +  On. 


nl  "  S 


TT, 


rh  +  (1  -  rlh, 
o _ 1 

m2  -  S  '  " 


*hohl 


2.  Plot  the  boundary  lines: 


-  n  +  rn 

o  o 


T  -■  h  +  ns 
1  1 


3.  Proceed  with  testing.  A  decision  is  reached  when  a 
boundary  is  crossed. 


1.6  EXPONENT  IAL  LISTRIBtTION 


The  exponential  distribution  has  been  observed  for  many  types  of 
complex  systems  mil  may  be  used  for  those  parts  and  systems  which 
are  so  complex  that  many  f  ypes  of  deterioration  with  different 
r a t  e s  a r e  < ? p e r a ble. 

It  is  a  special  case  of  the  Gamma  distribution  and  is  character¬ 
ized  by  a  constant  failure  rate.  The  confidence  interval  for  the 
mean  of  an  exponential  distribution  trom  a  random  s amp . e  of  times 
to- failure  can  be  obtained  by  using  the  Chi-square  distribution 
with  2r  degrees  of  freedom,  where  r  is  the  number  of  failures 
observed . 
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The  sequential  testing  procedure  for  exponential  distribution  is 
calculated  in  the  '"ol  lowing  manner. 

1.  Calculate: 

Slope  of  boundary  lines 


s 


T, 

LneM 

_ la _ 

1_ 

Ts  “  T, 


Intercepts  of  boundary  lines 


h 

o 


A 


h 


1 


B 


where ,  A 


Ln 


a 


1  -  S 


Average  sample  numbers  required  when  T  =  T,  ,  T  =  Ta ,  and 
T  =  s 


n 

o 


-  <y) 
-1)  + 


B  +  A 


n 


1 


BB  +  (1  -  8)A 
U-  f^>  ♦  Ln  |j- 


n 

s 


-BA 


[Ln 
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2.  Plot  the  boundary  lines: 

T  =  h  +  rs 

o  o 


3.  Proceed  with  testing.  a  decision  is  reached  when  a 
boundary  is  crossed. 

Several  different  distributions  are  used  t-  describe  the  para¬ 
meters  found  in  the  literature  of  reliability.  It  is  necessary 
for  the  engineer  to  have  a  working  knowledge  of  these  distribu¬ 
tions  n  order  to  understand  their  applications,  their  limita¬ 
tions,  and  the  time  and  labor  that  can  be  saved  by  their  proper 
use . 

When  statistical  tests  of  significance  are  set  up,  it  is  the 
responsibility  of  the  engineer  setting  up  the  test  to  consult 
with  the  statistician  in  order  to  determine  a  sequential  testing 
procedure  that  can  be  utilized  in  order  to  conserve  time  and 
money  and  reach  a  conclusion  that  will  provide  the  greatest 
amount  of  confidence  in  the  results. 


2  .  TEST  PROCEDURES  SUTMARY 

The  foregoing  discussion  has  considered  many  of  the  theoretical 
concepts  involved  in  testing  statistical  hypothesis  and  seme  of 
the  practical  problems  associated  with  these  concepts.  In  the 
forthcoming  section  actual  testing  procedures  will  be  summarized. 

2.1  CHI  SQUARE  (x?)  "GOODNESS  OF  FIT"  TEST 

This  procedure  is  used  to  determine  whether  or  not  a  set  of 
observed  values  is  consistent  with  a  uniquely  specified  density 
function.  This  uniquely  specified  density  function  is  the  null 
hypothesis.  The  alternative  hypothesis  is  that  the  true  density 
function  is  other  than  that  specified.  This  alternative  includes 
both  density  functions  of  other  forms  and  density  function  of  the 
same  form  as  that  specified  but  with  different  values  for  the 
paramete  . 

1.  All  possible  values  that  an  observation  can  take  on  are 

divided  into  N  classes,  each  class  being  some  "conveniently" 
chosen  interval  of  the  form  [aj  £  x  <  A1+1]  with  a,+1>  a,. 
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2.  The  probability,  pt  ,  that  an  observation  will  fall  into 
the  ith  class  is  computed  using  the  density  function, 

fP (x) ,  specified  by  the  null  hypothesis,  i.e., 

ai  +  1 

P,  -  f  f(x)  dx 
“  a: 

3.  The  expected  number,  e, ,  of  observations  falling  into  the 
ith  class  (for  each  value  of  i)  is  computed  using  the 
equation 

e.  =  n  •  pj 

where  n  is  the  number  of  observations.  It  is  required 
that  each  et  be  greater  than  five.  There  may  be  some 
regrouping  of  classes  in  order  to  fulfill  this  require¬ 
ment.  It  is  now  assumed  that  all  the  possible  values  of 
an  observation  has  been  divided  into  N  classes  and  that 
the  expected  number  in  each  class  is  greater  than  5. 

4.  Each  of  the  n  observations,  jx,  ,  Xg  ,  ......  xn  \  is  put 

into  its  proper  class.  Tne  number  of  observations,  0;  , 
in  each  class  is  then  computed. 

5.  The  quantity,  u,  is,  then  computed  where 


i=l 


6.  The  quantity,  u,  has  a  density  function  that  is  x2  with 
N-l  degrees  of  freedom.  The  desired  significance  level, 
a ,  is  now  used.  Entering  the  x2  tables  with  N-l  degrees 
of  freedom  and  ry ,  a  value,  x2/  such  that 

P(XS  >  Xo)  =  »  or  P(x2  «  Xo)  =  1  -  a 
is  obtained. 

7.  If  u  -  Xo/  the  data  is  said  to  be  consistent  with  the 
null  hypothesis  at  the  a  significance  level;  if  u  ^  Xo 
the  data  is  said  to  be  inconsistent  with  the  hypothesis 
at  the  :y  significance  level,  i.e.,  the  null  hypothesis  is 
rejected. 
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The  following  point  regarding  the  x2  "Goodness  of  Fit"  test  may 
be  noted.  Suppose  that  the  density  function,  f(x),  has  k(k<N-l) 
parameters  not  specified  oy  the  null  hyDothesis.  One  could  use 

the  observations  {xx . .  xn \  to  provide  estimates,  via  the 

maximum  likelihood  estimator,  of  the  k  parameters.  These  estim¬ 
ated  values,  |  ,  e2  9V  }-,  could  then  be  substituted  into  the 

density  function  and  expected  values  computed  as  was  done  pre¬ 
viously.  The  statistic,  u,  now  has  a  density  function  that  is 
vs  with  N-l-k  degrees  of  freedom. 

Examples : 

1.  A  die  was  cast  360  times  with  the  following  result. 

Result  of  toss  123456 

Frequency  59  65  52  63  67  54 

Are  the  data  consistent  with  the  null  hypothesis  that  the  die  is 
true  at  the  0.05  significance  level. 

Step  Is  H0  spx  =  p2  =  p3  =  p4  =  p6  =  p6  =  | 

e  =  =  e2  =  e,  =  e4  =  eB  =  eR  =  ^  x  360  =  60. 

Step  2s  xi  (5  degrees  of  freedom,  a  =  0.05)  =  11.1 
Step  3  s 

(60-59) 2  (60-65) 2  (60-52)2  (60-63)2  ,  (60-67) 2 

U  60  +  60  +  60  +  60  60 


(60-54) 2 
+  60 


to  [lt 


25  +  64  +  9  +  49  +  36]  = 


184 

60 


3.07 


Step  4s  u  *  Xo-  Hypothesis  is  retained. 

2.  A  Monte  ^arlo  method  is  used  to  generate  times-to- failure 

X  x  1 

having  a  density  function,  \e  '-here  X  =  — .  Two 

thousant  such  times  have  been  generated.  The  class  data 
are  presented  in  Table  10- 1*  Each  class  interval  repre¬ 
sents  3  time  units  from  0-60  with  the  last  interval  con¬ 
taining  all  observations  greater  than  60. 

Are  the  2000  observed  values  consistent  with  H0:f(x)-\e 
where  X  =  1/36  at  the  0.10  significance  level? 
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TABLE  10.1 


SUMMARY  OF  MONTE  CARLO  SAMPLING 
FROM  DENSITY  FUNCTION,  ~  e"  X 


Class 

Interval 

Number 

Class 

Interval 

De  f inition 

Observed 

Nuit'iber  in 
Class,  0, 

Expected 
Number  in 
Class,  e, 

(e,  -  0,  )? 

1 

o^-t'-  3 

147 

159.9 

1.0407 

2 

3  -t<6 

17  0 

147 . 1 

3 . 5650 

3 

6‘  t<9 

131 

135.4 

0.1430 

4 

9^t< 12 

117 

124.5 

0.4518 

5 

12  -;t<  1  5 

113 

114.6 

0.0223 

6 

1  5^t<  18 

110 

105.4 

0.2008 

7 

18  •  t'  2 1 

99 

97 .0 

0.0412 

8 

2  let-  24 

87 

89.2 

0.0543 

9 

24- 1''27 

81 

82.1 

0.0147 

10 

o 

•-'n 

r- 

CM 

81 

75.5 

0.4007 

11 

30et''  3  3 

83 

69.5 

2.6223 

12 

33^t‘:  36 

73 

63.9 

1.2959 

13 

36  t<35 

61 

58.8 

0.0823 

14 

39^t<42 

48 

54.1 

0.6878 

15 

42 -  f  45 

40 

49.8 

1.9285 

16 

45‘:t'-48 

54 

45.8 

1.4681 

17 

48'Jt*'51 

35 

42.2 

1.2284 

18 

51  "t''  54 

30 

38.8 

1 .9959 

19 

54'^t"-57 

31 

35.7 

0.6188 

20 

57 et<60 

33 

32.8 

0.0012 

21 

t‘  60 

376 

377 .8 

0.0086 

2000 

2000 

17 .8723 

4 

4  V 
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Step  1. 


1_ 

36 


x 


1_ 
12 

_1_ 

et  =  2000  ( 1  -  e  12 )  =  159.9 


(a) 


1 

r  i_ 


36 


x 


J 

o 


■  :  1 


36 


(b) 


ea  =  2000  (e 


12 


-  e 


)  =  147.1 


etc . 

The  values,  et  ,  are  also  presented  in  Table  10.1. 
Step  2.  Xo  (20  degrees  of  freedom,  a  =  0.10)  =  28.4 


21 

Step  3.  u  =  Y  — 1  ~  ■°1"-  =  17.87 

L  ei 

i=  1 

Step  4.  u  <  Xo  30  null  hypothesis  is  not  rejected. 

3.  If  H0:t(x)  =  \e  did  not  specify  X,  then  step  (1)  would 
have  used  the  data  to  obtain  an  estimated  value,  X,  for 
X.  The  value,  X,  rather  than  X  =  1/36  would  then  be  used 
to  compute  values  for  e. .  The  only  other  change  would 
then  be  determining  Xo  using  the  x3  tabular  values  for  19 
degrees  of  freedom  rather  than  20. 

Before  discussing  other  test  procedures  there  are  a  number 
of  points  illustrated  by  the  above  examples  that  should 
be  explicitly  stated. 
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In  the  second  example,  a  >  f',  value  of  28.4  ( -,  ■=  0.10  and 
20  degrees  of  freedom)  was  used.  If  the  null  hypothesis 
is  true  then  the  odds  are  9:1  against  obtaining  a  value 
of  u  greater  than  *(- .  If  a  set  of  observations  produces 
a  value,  u1 ,  that  is  greater  than  xf  we  have  one  of  two 
choices.  One  choice  is  to  assume  that  the  null  hypothesis 
is  true  despite  the  fact  that  an  "unlikely"  event  (9:1 
odds  against  it)  has  occurred.  The  second  choice  is  to 
reject  the  null  hypothesis.  This  second  choice  represents 
the  attitude  taken  in  the  development  of  the  concepts  of 
testing  statistical  hypothesis. 

The  next  point  concerns  the  observed  numerical  value  of  u 
in  the  second  example.  One  may  remember  that  Up^g  -  17.9. 
Furthermore ,  H0  was  not  rejected  since  u 0ks  <  =  28.4. 

Suppose  u0bs  were  28.0  rather  than  17.9.  Statistics  makes 
no  distinction  between  these  two  values.  The  rejection 
region,  [28.4,  »] ,  has  been  selected  to  yield  a  given 
significance  level.  If  one  decides,  after  calculating 
28.0  as  the  value  for  u,  to  reject  H,,  ,  he  is,  in  effect, 
changing  the  significance  level.  He  is  not  acting  in 
accordance  with  his  pre-calculation  stated  desires.  Thus, 
while  the  two  values  indicate  different,  degress  of  con¬ 
sistency  between  theory  (expected  values)  and  practice 
(observed  values),  both  degrees  of  consistency  are  within 
the  limit  specified  y  the  stated  significance  level. 

In  a  similar  vein  remember  that,  in  general,  acceptance 
or  rejection  of  the  null  hypothesis  is  a  matter  of  compari 
son  of  degree  of  consistency  between  theory  (or  theories) 
and  practice  -  it  is  not  a  100%  positive  statement  con¬ 
cerning  the  truth  of  the  null  hypothesis  or  the  alternativ 

Indeed,  to  be  extremely  literal,  it  does  not  concern  the 
term  "truth”  at  all;  rather,  it  concerns  degrees  of  con¬ 
sistency  between  a  set  of  data  and  two  hypotheses  as  well 
as  a  selection  of  some  limiting  degree  of  consistency  to 
determine  which  of  two  hypotheses  is  to  be  preferred. 

2.2  THE  "NORMAL"  TEST 

Case  1.  A  sample  of  m  observations  is  taken  from  a  n(x;y.,r») 
population  where  a  -  is  a  specified  number. 

Test  H0  :  -  b  (b  a  specified  number)  against  Hj  >  b 

at  a  significance  level  of  a. 


r 
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Step  1:  Determine  a,  from  cumulative  Normal 


tables  such 

that  Nfa-,;  0,  1) 

=  1  -  or . 

Some  typical  v 

a  lues  of  a  ,  are  : 

y  1  —  j 

a.  o 

l 

1  -  3  a} 

0.20  0.80 

0.842  0.05 

0.95  1.645 

0.10  0.90 

1  .282  0.01 

0.99  2.326 

Step 

2: 

Calculate  X  (arithmetic  average) 

of  observations 

Step 

3  : 

Calculate  ~ 

0 

-  b 

/  r-  r 

*.'m 

Step 

4 : 

If  r  >  ai  reject  He  in  favor  of 

H ■ ;  otherwise 

Ho  is  retained 

• 

Case  2.  Same  as  Case  1,  but  =  b  is  to  be  tested  against 

FI  :  L.  •  b  at  a  significance  level  of  a. 

Step  1:  Same  as  Steps  1,  2,  and  3  of  Case  1. 

Step  4:  If  r  a^,  reject  in  favor  of  ,  otherwise  H 
is  to  be  retained. 

This  is  called  a  one-tailed  test.] 

Case  3.  Same  as  Case  1,  but  Hc :  n  -  b  is  to  be  tested  against 
H j  :  u  *  b  at  a  significance  level  of  a. 

Step  1.  Compute  a  =  1  -  j 

Step  2.  Determine  ar  from  the  equation  N(aff  *  0,  1) 
p ( y  s  a7)  using  Cumulative  Normal  tables. 


Typical  values 

of  a-j, 

are : 

cv  l-(f) 

ao 

or 

l-(f) 

a 

ar 

0.20  0.90 

1.282 

0.05 

0.97  5 

1.960 

0.10  0.95 

1.645 

0.01 

0.995 

2.576 

Calculate  r  = 

X  -  b 

a  /  , — 

o  ym 


| 
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Step  4.  If  r  '■-a  or  r  >  a  reject  Hr  ir.  favor  of 

Ht ;  otherwise  H0  is  retained. 

[This  is  called  a  two-tailed  test.] 

Case  4.  A  sample  of  m,  observations  is  taken  from  a  nfx;^,?,) 
population  where  a*  is  a  specified  number. 

A  second  sample  of  ma  observaticns  is  taken  from  a 
n(x;u s,o2)  population  where  cra  is  a  specified  number. 

Hc  : u, ^  =  ^2  is  to  be  tested  against  Hj  >  ui  at  a 
significance  level  of  r» . 

Step  1.  Determine  aa  such  that  N(a^;  0,  1)  =  1  -  a  as 
was  done  in  Case  1. 


Step  2.  Compute  r  = 


where  X.  is  the  arithmetic  average  of_the 
observations  in  the  first  sample  and  is  the 
arithmetic  average  of  the  observations  in  the 
second  sample. 

Step  3.  If  r  >  a-j  reiect  Hr ;  otherwise  retain  H0 . 

Case  5.  Same  as  Case  4  but  H0:u,-  =  u ,  is  to  be  tested  against 

H;  *  uj  at  a  significance  level  of  ■> . 

a ^  such  that  N(a>;  0,  1) 

3. 


reject  H, :  i f 


Step  Is  Compute  o'  =  1  -  j  and 
-o  as  was  done  in  Case 


-  X 

Step  2  s  Compute  r  =  — * - L- 


m. 


m. 


Step  3s  If  r  <  -a-,',  or  r  >  a, '  , 
a  ^  r  ^  aa  accept  Hc . 


X,  -  X, 


m. 


n\n 


It  may  be  noted  that  rejection  regions  have  been  defined  by  in¬ 
equalities  of  the  form  r  >  a  with  acceptance  when  r  <  a.  Chang¬ 
ing  rejection  inequalities  to  r  i  a  and  acceptance  when  r  <  a 
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does  not  require  any  chanyes  in  previous  steps. 

The  following  cases  require  a  density  function  known  as  the 
"Student  t"  and  the  procedures  are  called  "t  tests". 

In  the  six  "Normal"  tests  above  it  was  always  assumed  that  the 
standard  deviation  values  were  known,  ^hen  the  standard  deviat¬ 
ions  are  unknown,  estimates  of  c  must  be  obtained  from  the  sample 
observations.  If  estimates,  s,  of  a  are  used,  then  the  "Normal" 
test  is  not  applicable  and  the  "t"  test  must  be  used  instead 
The  "t"  distribution  is  used  in  the  same  fashion  as  the  "Normal" 
for  both  one  sided  and  two  sided  tests. 

Values  of  the  cumulative  (F(d)  =  P(x  <  d)  Student  distribution 
are  presented  in  Figure  10-55.  it  may  be  noted  that  to  obtain 
a  particular  tabular  entry,  two  numbers  are  required,  a  value 
for  n(left  most  column)  and  a  value  for  F(d).  The  entry  found 
is,  then,  the  appropriate  value  for  d.  (The  value  of  n  is  called 
the  number  of  degrees  of  freedom.)  As  an  example,  for  19  degrees 
of  freedom  and  F(d)  =  0.975,  the  required  value  for  d  is  2.093. 

We  shall  denote  this  value  obtained  from  the  table  as  t0  975  19  ~ 
2.093. 

y 

The  following  notation  will  be  used  in  describing  various  "t 
Test"  procedures. 


1.  Tt  will  denote  the  sample  arithmetic  average.  In  parti¬ 
cular,  if  there  are  m  observations 
m 
r 

L  x> 

X  =  — 

m 

If  more  than  one  sample  is  involved,  a  subscript  on  X  will 
be  used  to  indicate  the  applicable  sample.  The  jth  obser¬ 
vation  in  the  ith  sample  will  be  denoted  by  x, 3 

m, 

V  x 

L  1  3 

—  i  *  1 

thu-  X,  =  -* - 

m, 

where  m,  denotes  the  number  of  observations  in  the  ith 
sample . 
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2.  T  will  denote  the  total  of  the  observations 


-  T  - 
X  -  —  or  mX  =  T 
m 


X,  =  —  or  m,  X.  =  T. 

m. 


3.  S'  will  denote  the  sample  standard  variance 


m 

1.1-1 

j  =  l 


(x. -x) 


m 

1  r  r  • 

m- 1  L. 

j=l 


T' 


l  r~  —  _  1  rr  t.  0  o 

Sf  =  — 7  ,  (x.-xv  ---  ■  r  !  )  x;  .  -  —  ! 

m - 1  nr.,  - 1  L  m ,  j 

i  =  i  j  = l 


4.  S  =  JS':  (or  S.  =  Js\  )  is  the  sample  estimate  of  the 
standard  deviation! 

Case  6.  A  simple  of  :r.  observations  is  taken  from  a  n(xj„,  -) 

population.  H  : _  -  b  is  to  be  tested  against  H,  :„>b  at. 
a  significance  level  of  . 


Step 

1. 

Determine 

t 

,  ,  us  inn 

1  -  ,  m  -  1 

Fi 

oure  10- ^  5 . 

Step 

«£.  • 

Calcu late 

S 

and  then  r  - 

X 

s- 

-  b 

m 

Step 

3  . 

If  r  ’  t 

~  't 

,  r elect  H. 
,m-l 

; 

otherwise  H. 

is  retai ned 

Case  7.  Same  as  Case  6  but  H,  :  :.-b  is  to  be  tested  against 
H.  b  at  a  significance  level  of  . 


Step  1.  Determine  t  as  above. 

I  —  r  f  1 


Step  2.  Calculate  S  and  ♦•.hen  r 


X  -  b 


Step  '.  If  r  -t 


1-  i,m-l 


reject  H  ;  otherwise  H.  is  retair.e 
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Case  8.  Same  as  Case  6  but  H.  :..~b  is  to  be  tested  against 
H, *  b  at  an  a  level  of  significance. 


Step 

1. 

Compute  ->  -  1  -  ->/ 2 . 

St  ep 

2  . 

Determine  t  ,  ,  using  Figure  10-26. 

•>  (m-l 

Step 

3  . 

X  -  b 

Compute  S  and  then  r  =  ,  - 

S/  m 

Step 

4. 

If  r  '  -t  ,  ,  or  r  t  ,  ,  reject 

a  ,  m-1  ->  ,m-l 

H-  ;  otherwise  H,  is  retained. 

Case  9.  A  sample  of  size  m,  is  taken  fi  >m  an  n(x;„.  ~)  population. 
A  second  sample  of  size  nu  is  ^.aken  from  a  different 
n(x;^r,')  population.  It  ic  assumed  that  ‘"he  ~  of  these 
two  populations  are  equal  and  that  this  common  value  is 
to  be  determined  us;ng  sample  values. 


H.  is  to  be  tested  against  H.  : __  >  „,  at  a 

significant  level  of  -> . 

Step  1.  Determine  t, 

1-  , ,  m.  +  rru  -  2 


Step  2 
Stop  3 


S  t.  e  p  4 

S  t.  o  p  *> 


Compute 

m .  •  m.. 

m ,  +  rru, 

Compute 

m, 

m. 

1 

r 

r~ 

■  (X.  . 

-  X)  -  r 

i — 

'  ( X, 

,  -  X )  '  ! 

v">  - 

nu  -mtu 

-2 

1  1 

i  -  1 

m . 

rn.. 

i 

r 

r- 

'  (x:  . 

T.  ■' 

|  ( Xp  •. 

T,  :  1 

m,  +r,w. 

'i 

- 

m  ^ 

TTU,  j 

]-l 

1  1 

Compute  r 

X 

...  -  X  - 

S.  e 

If  r  >  t 

1  -  , 

rn,  tnv  -2 

H.  is 

ret  octed 

rn  favor 

other'-- is 

e  H. 

is  retained. 
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Case  10.  Same  as  Case  8,  but  a-  is  to  be  tested  against 

H.  at  a  significance  level  of 

Step  1.  Compute  a  '  -  l  -  ~ 

Step  2.  Determine  t.  ,  .  using  Figure  10-55. 

a  ^^+70-3-2 

Step  3.  Same  s  Steps  2,  3,  and  4  of  Cnse  3. 

Step  4.  If  r  <  -t  /  ,  or  if  r  >  t  /  „  reject  H, 

<y  ,m-,i+mr-2  a  ,n, +mj-^ 

in  favor  of  H,  ;  otherwise  H=  is  retained. 

The  next  level  of  abstraction  in  tnis  series  of  test  cases  would 
be  the  removal  of  the  assumption  that  the  standard  deviations  of 
the  two  populations  involved  are  equal.  Unfortunately,  this  re¬ 
moval  leads  to  a  discussion  quite  beyond  the  scope  of  this  course. 
The  reader,  interested  in  pursuing  this  problem,  is  referred  to 
any  standard  test  on  mathematic!  statistics.  However,  using  the 
statisitic,  r,  where 

r  =,  *3- 

rSTS 

V  mq  rte 

and  the  "Student  t"  table  with  +1^-2  degrees  of  freedom  is 
the  custanary  procedure  followed  ^r.  this  case. 

The  cases  involving  the  student  t  distribution  discussed  above 
h  ve  all  assumed  an  initial  Normal,  n(x;u,<?)  population.  (Indeed, 
for  samples  of  sizes  greater  than  30,  the  normal  distribution 
may  be  used  to  determine  regions  of  rejection  even  though  a2  is 
estimated  from  the  data).  However,  the  t.  distribution  is  appli¬ 
cable  in  a  wide  variety  of  situations  in  which  the  underlying 
population  is  not  Normal. 

To  illustrate  this  last  statement  we  consider  the  following 
experiment  which  was  repeated  100  times. 

All  jacks,  queens,  and  kings  were  removed  from  a  standard  deck 
of  cards.  The  remaining  cards,  1  -  10,  represented  a  uniform 
distribution,  P(x=k)  =  1/10  for  k  =  1,  .....  10.  A  sample  of 
size  5  -was  drawn  from  this  deck  in  such  a  way  that  the  probabil¬ 
ity  of  drawing  any  denomination,  (1-10),  was  the  same  from  draw 
to  draw. 


Now,  the  true  mean  of  the  original  population  was  known, 


5.5. 
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The  sample  of  size  5  was  used  to  test  H. 
at  a  10  h  significance  level  as  follows? 


> ,  against  H,  :  ^  5 . 5 


(1)  a 


=  1 


0.9! 


*0.95,4  2-lj2 


(3)  S  ~  4  Mx'  ~  X)" 
i=i 


(4)  r  - 


X 


S/,'5 


(5)  If  r  <  -  2.132  or  r 
H0  was  accepted. 


2.132.  H.  was  reiecred;  otherwise 


According  to  the  development  of  the  theory,  H3  should  be  rejected 
10%  of  the  time.  In  100  trials  the  actual  number  of  rejections 
was  9,  i.e.,  the  observed  proportion  of  rejections  was  9.0%,  a 
fair  degree  of  approximation  since  the  parent  distribution 
was  far  from  normal  and  five  is  a  small  sample  size. 


As  long  as  the  density  function  of  X  is  fairly  symmetric  about 
the  mean  of  the  parent  density  function,  the  "t  test"  can  be  used 
even  though  the  parent  population  is  not  Normal,  n(x;u,a). 


Case  11.  A  group  of  m  identical  items  is  to  be  put  on  trial 
until  all  m  fail. 

Each  item  is  assumed  to  have  a  xe  density  function 
for  time-to- failure.  Of  course,  the  values  of  x  assoc¬ 
iated  with  each  item  are  assumed  to  be  equal.  H0:\=x0 
is  to  be  tested  against  iX  <  >  „  at  an  at  level  of 
significance. 


Step  1. 


Step  2 . 


Enter  the  x2  taole  with  2m  degrees  of  freedom  and 
with  a  and  determine  a  value,  x2  »  such  that 
P(X2  <  Xo)  -  a  or  P(X2  >  x»)  =  1  -  a. 


Determine  T0 


*  XaL 

2X0 


Step  3.  Calculate,  T,  where  T  is  the  total  of  the  observed 
times-to- failure  for  the  above  m  items. 


S‘ep  4.  If  T  <  T0  reject  H0  in  favor  of  H,  ;  otherwise  retain  H0 . 


:  Wawis 


Case  12.  Same  as  Case  10,  but  now  the  total  operating  time  of 
the  m  trials  is  fixed  in  advance  to  be  T0 . 

NOTE:  THE  TRIAL  TIME  ASSOCIATED  WITH  •  EACH  ITEM  IS  NOT  BEING 

FIXED  IN  ADVANCE. 

Step  1.  The  experiment  if  performed  and  the  number  of 

failures,  tk,  occurring  during  the  test  is  noted. 

Step  2.  A  value  of  Xo»  as  in  Case  10,  is  determine  entering 
the  tables  with  2k  +  2  degrees  of  freedom  and  with 
the  value  of  ». 

Step  3.  If  T0'  <  reject  H0  in  favor  of  H.  ;  otherwise  H3 
is  retained. 

Case  13.  Same  as  Case  8  but  now  the  experiment  is  terminated 

when  the  kth  failure  occurs;  k  is  a  specified  integer. 
(This  is  one  method  of  truncating  the  experiment) . 

Step  1.  Determine  using  2k  degrees  of  freedom  and  y  as 
was  done  in  Case  10. 

Step  2.  Compute  total  operating  time,  T,  of  the  m  items  in 
the  test. 

k 

T  =  ^  tj  +  (m-k)t!c 

i=l 

2 

Step  3.  If  T  <  reject  H0  in  favor  of  H, ;  otherwise 

2a  o 

accept  H0 . 

3 '  DESIGN  OF  EXPERIMENTS  AND  ANALYSIS  OF  VARIANCE 

In  a  previous  chapter  we  have  considered  in  some  general  terms 
a  concept  called  a  test  plan.  In  this  section  we  consider  speci¬ 
fic  statistical  techniques  that  may  be  used  within  this  test  plan, 
in  particular  that  area  of  statistics  called  design  of  experiments 
and  the  methodology  -  analysis  of  variance  -  used  therein. 

Historically,  the  design  of  experiments  was  first  used  extensively 
in  agricultural  experiments.  The  standard  technique  of  "differ¬ 
ences  between  two  means";  i.e.,  the  "t"  test  had  been  used  to¬ 
gether  with  data  from  a  "treated"  and  "control"  group  to  determine 
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if  the  "treatment"  really  had  an  effect.  However,  things  got 
more  complicated.  Numerous  factors  such  as  type  of  seed,  type 
and  amount  of  fertilizer,  amount  of  water,  had  to  be  considered 
simultaneously.  The  use  of  paired  comparisons  meant  inefficient 
use  of  resources  as  well  as  inefficient  production  of  information 
based  on  the  data  gathered  from  these  tests.  With  the  advent  of 
the  development  of  Design  of  Experiments  and  the  associated  use 
of  Analysis  of  Variance,  a  method  of  making  a  number  of  simultan¬ 
eous  comparisons  with  one  testing  procedure  was  available.  This 
method  meant  net  only  that  the  data  from  a  test  could  be  treated 
more  efficiently,  but  that  the  test  itself  could  be  designed  to 
yield  the  same  information  w'ith  the  use  of  fewer  resources,  i.e., 
the  resources  were  used  mere  efficiently. 

There  are  complete  half-year  graduate  level  courses  in  design  of 
experiments.  Clearly,  we  cannot  develop  this  area  so  completely 
here.  Therefore,  we  shall  consider  only  the  concepts  involved  in 

this  discussion. 

3.1  APPLICABILITY  OF  USE  OF  EXPERIMENTAL  DESIGN 


Experimental  design  is  used  to  determine  whether  or  not  the 
effects  of  "factors'  are  significant.  "Factors"  can  involve  the 
external  environments  of  operation  as  humidity,  temperature,  salt 
air  concentration,  etc.  "Factors"  may  also  involve  the  opera¬ 
tional  conditions  of  applied  voltage,  or  pressure  head,  or  RPM  - 
as  appropriate.  Obviously,  this  technique  could  be  used  when  it 
was  thought  that  the  effects  of  such  factors  on  reliability  were 
both  strong  and  deleterious.  However,  while  the  design  of  exper¬ 
iments  does  represent  an  efficient  means  cf  gaining  desired  in¬ 
formation,  its  efficiency  is  directly  proportional  to  the  amount 
of  knowledge  as  to  the  factors  which  are  possibly  important  in  a 
particular  problem.  Use  of  this  technique  presumes  the  absence 
of  the  "shotgun  approach"  to  finding  "causes"  of  failure.  Thus, 
the  statistician  is  dependent  on  the  engineer  to  define  the 
"factors",  while  the  engineer  relies  on  the  statistician  to  pro¬ 
vide  "efficiency." 

As  will  be  seen,  the  use  of  this  technique  can  still  require  a 
considerable  expenditure  of  time  and  funds,  Therefore,  its  use 
in  a  development  program  is  usually  limited  to  critical  hardware 
items.  Furthermore,  there  should  be  a  reasonable  amount  of  en¬ 
gineering  confidence  that  the  possible  important  factors  inducing 
the  problem  to  be  studied  has  been  delineated  before  such  a  de¬ 
sign  is  implemented. 
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Finally,  in  order  to  consider  a  particular  factor"  in  the  design 
of  the  experiment,  it  must  be  possible  to  provide  a  reasonable 
amount  of  control  over  the  values  that  the  "factor"  may  take  on 
during  the  course  of  the  experiment,  i.e.,  the  "factor"  values 
must  be  reproducible. 

3.2  ANALYSIS  OF  VARIANCE 


While  the  arithmetic  associated  with  analyzing  data  from  experi¬ 
ments  that  have  a  complex  design  can  be  very  complicated,  the 
fundamental  arithmetic  identity  is  an  extremely  simple  one.  We 
shall  provide  an  illustration  of  this  identity  first. 

The  general  formulation  of  the  arithmetic  identity  to  be  used  can 
be  written  as  follows: 

n  n 

V  (Xt  -  a)2  =  [  y  (X4  -  X)2]  +  n(X  -  a)3 
i=l  i=l 

Let  us  see  hew  this  equation  applies  in  a  rather  simple  case. 

There  are  two  sets  of  observations  -  each  set  with  four  observa¬ 
tions.  Observation  values  in  the  first  set  are  denoted  by  Xa ! , 
x13  ,  Xl3,  X14.  For  our  illustration  Xr  j  ~  1,  Xie  =  2,  Xj ,  =  3, 

Xl4  *  4.  The  second  set  of  values  are  X^  =  8,  Xg3  =  9,  X^  =  1C, 
and  Xg  4  —  11. 

(1)  The  eight  observations  considered  as  a  single  grouf.  has 
a  mean  of  six  and  we  write  X  (the  grand  or  overall  mean) 

=  6.0. 

(2)  The  sum  of  the  six  squared  deviations  from  this  "grand 
mean"  of  6.0  is  108.0,  i.e., 

I  4 

y  y  (x4  3  ~  x)3  =  io8.o. 

j=i  i=i 

This  sum  of  overall  observations  of  the  squared  deviations  from 
the  grand  mean  is  called  the  "Total  Sum  of  Squares". 

(3)  The  mean  in  the  first  group  is  'Xv_,=  2.5  and  in  the  second 
group  the  mean  is  1?,  -  9.5.  Now.)  nfXj  -  X)2  is  called  the 
"between  group"  sum  of  squares. 


The  value  of  this  "between  group"  sum  of  squares  is  the 
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value  of  4(2.5  -  6):>  +  4(9.5  -  6)?;  -  4  [(3.5)*  +  ( -  3 . 5 ) :  3 

=  4  [12.25  +  12.25]  -  4  [24.5]  =  98.0. 

(4)  Considering  the  first  group  of  four  observations  as  a 
unit,  they  have  a  mean  of  2.5  and  the  sum  of  squared 
deviations  from  this  group  mean  is  5.0.  This  is  the 
"within  group"  sum  of  squares  for  the  first  group.  The 
second  group  has  a  mean  of  9.5  and  a  "within  group"  sum 
of  squares  of  5.  The  total  "within  group"  sum  of  squares 
is,  then,  5  M  or  10. 


(5)  The  total  sum  of  squares,  108,  has  thus  been  partitioned 
into  (a)  the  "between  group"  sum  of  squares  98.0,  and 
(b)  the  "within  group"  sum  of  squares  of  10. 


2 


i=l  j-'l 


f  V(v 

\  L  1 


i=  1  j  =  l 


4  £  (X,  -  X)» 
i~l 


or,  in  general, 


k  nt 


i=l 


k  rq 


[  l  ( 

x-1  j  =  l 


k 

X)  ]  +  Y  n,  (Xs  ~  X)a 
i=l 


where  there  are_k  different  groups,  n,  observations  in 
the  ith  group,  X  is  the  grand  mean  and  Xj  is  the  mean 
in  the  i^h  group. 


As  a  final  note,  the  total  "within  group"  sum  of  squares 
is  sometimes  referred  to  as  the  experimental  error. 

3.3  THE  BASIC  DESIGN  OF  EXPERIMENTS  MODEL 


A  One  Factor  Experiment 

For  the  purposes  of  this  discussion  it  shall  be  assumed  that  we 
have  n  observations  in  each  of  k  groups  -  a  total  of  n  x  k 
observations.  The  n  specimens  in  a  group  might  be  n  transistor 
operating  at  a  common  ambient  temperature.  The  differences  between 
the  group?  are  different  ambient  temperatures.  (In  standard 
terminology  -  the  different  groups  represent  different  vreatments ) . 
The  observations  might  be  times-of- failure. 

It  is  assumed  that  the  observation,  X, t  ,  associated  with  the  jbh 


10-34 


specimen  in  the  i^  group  can  be  written  as 
Xi  3  =  H  +  bi  +  ei  j 

where  \i  is  called  the  true  overall  mean,  bt  is  called  the  true 
treatment  effect  and  i  is  the  deviation  of  the  observation 
from  the  true  group  mean,  u  +  bj  .  It  is  furthermore,  understood 
that  u  is  such  that 

k 

l b*  -  °- 

i=l 

Now,  the  overall  sample  mean,  X,  is  an  estimator  for  p;_the  sample 
group  mean,  Xj  ,  is  an  estimator  for  p  +  bj  ,  thus,  Xj  -  X  is  an 
estimator  for  bt  . 

It  is  now  assumed  that  the  es  i  all  represent  independent  observa¬ 
tions  from  the  same  Normal  density  function  n(X,  ;  0,  cr)  .  The 
fact  that  e,  i  ' s  are  independent  implies  that  (1)  repair  does  not 
influence  the  t ime-to-the-next- fai lure  and  (2)  wear-out  is  not  a 
factor  in  the  successive  failures.  The  assumption  that  the  es  . ' s 
come  from  the  same  density  function,  n(X;  0,  a),  is  used  as 
follows . 

Now,  x4  j  -  Xjs( |i>  +  b,_+  etl)  -  (u  +  bj  +  e,' )  =  e,  j  -  e/  where  e/ 
is  the  deviation  of  X!  from  p  +  b,  .  (Note  e,'  is  a  function  of 
ei  l  *  e18...,e1/B  for  each  value  of  i.). 

k  n 

Thus'  I  '  X‘  )aj-  is  a  function  only  of  the  e,j's.  This 

i=l  j  =  i 

"within  group"  sum  of  squares,  then,  may  be  used  to  estimate  a3. 

Now,  X!  -  X  =  4  +  bj  +  e,'  -  (p  +  e")  or  X,  -  X  =  bt  +  (e/  -  e") 
where  e*  is  a  function  of  all  the  e,  }  .  Thus 

k  k 

pY  (X,  -  3t)a  =  nY  (bt  +  e,'  -  e*)a  This  “between  group"  sum  of 
i=l  i=l 

squares  therefore  involves  the  bj  's  an^  the  es  j’s. 

The  null  hypothesis  in  this  design  is  that  all  the  bs  's  are  equal 
to  zero.  Under  the  null  hypothesis,  the  "between  group”  sum  of 
squares , 
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k 

"I  <*. 


-  x)s 


k 


-  e")8 


becomes 


k 


ft  \ ; 

-  e  )' 


i=l  i=l  i  =  l 

which  is  a  function  of  the  e,,^  only.  Indeed,  under  the  null 
hypothesis  the  "between  group"  sum  of  squares  can  also  be  used 
to  estimate  a2  . 


Let  sf  denote  the  estimate  of  using  the  within  group"  sum  of 
squares.  Let  S2  denote  the  (independent  of  S2)  estimate  of  a2 
using  the  "between  group"  sum  of  squares.  Under  the  null  hypo¬ 
thesis,  the  ratio,  Sf/'S2,  represents  the  ratio  of  two  independent 
estimates  of  the  same  quantity,  a3.  The  ratio  should  have  values 
close  to  1. 


On  the  other  hand,  if  the  null  hypothesis  is  not  true,  i.e.,  at 
least  one  of  the  bt  1 s  is  not  zero,  then  sf  should  be  greater  than 
S2 .  Thus,  the  region  of  rejection  of  the  null  hypothesis  is 
sf/s8  >  F0  where  th  i  constant,  F0  ,  has  to  be  determined.  The 
manner  of  datermininq  this  value  of  F0  and  the  means  of  converting 
the  "within  roup1  and  "between  group"  sums  of  squares  to  estimates 
of  a3  wlII  be  dis  ussed  in  the  next  section,  the  F  Test. 

3.4  THE  S’  TES. 

We  shall  suit^arize  the  discussion  of  the  previous  section  by  an 
analye  is-oi'-var  iance  table  for  a  one  factor  experiment.  The  first 
t'*'o  columns  represent  the  partitioning  of  the  total  sum  of  squares. 
The  third  c  'lumn,  degrees  of  freedom,  is  a  ditisor  which  converts 
each  of  the  sum-of-squares  parts  to  an  estimator  for  c8 .  The  last 
column  represents  the  observed  value  of  the  F-ratio.  (see  table 
10.2). 


To  obtain  the  critical  value,  F0 ,  such  that  tine  null  hypothesis 
is  rejected  if  Sf/S2  >  F0 ,  the  F  table,  here  presented  in  Figure 
10-55  is  used.  Three  values  are  required  to  enter  this  table. 

One  is  the  number  of  degrees  of  freedom,  mf  associated  with  the 
numerators,  S2  .  Here,  m  is  k  -  1 .  The  second  is  the  number  of 
degrees  of  freedom,  N,  associated  with  the  denominator,  S2  .  In 
this  case  N  =  i  x  (n-1)  .  The  third  is  the  significance  level,  or. 
In  Figure  10-55  entry  is  made  using  1  -  a. 

To  illustrate  this  particular  case  we  consider  four  groups,  Gx , 

G3  ,  03  ,  G4  ,  with  3  observations  in  each  group 
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2 

3 

4 


TOTAL 

MEAN 


3 

4 

5 

12 

4 


4 

5 

6 

15 

5 


5 

6 
7 

18 

6 


=  _  9+12+15+18 
X  12 


—  =45 
12  4*5 


(a)  The  total  sum  of  squares,  ^  j  -  X)8  ,  can  be  computed  by 
the  formula  *”* 


Where  T  is  the  grand  total  of  all  the  observations. 

V  ,  =  4  +  9  +  16  +  9  +  16  +  25  +  16  +  25  +  36  +  25  *  36  + 
49  =  266 


T3  =  (9  +  12  +  15  +  18 ) 3  _  ( 54) 3  _  27  x  27  = 


n  k 


12 


12 


9  x  27  =  243 


(a-1'  T^tal  SS  (sum  of  squares)  =  266  -  243  =  23 
(a-2)  Number  ol  degrees  of  freedom  =  11 

(b)  Within  group  SS 

For  each  of  the  four  groups,  X<x, : 
Total  for  the  four  groups  is  8 . 

(b-1)  Within  Group  SS  =  8 

(L~2)  Number  of  degrees  of  freedom  *  k  x  (n-1)  *4x2*8 

( c )  Between  Group  SS 

n[  -  S)8]  =  3  x  f  (3  -  4. 5)a  +  (4  -  4.5)8  +  (5  -  4.5): 

+  (6  -  4.5)8] 
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=  3  x  r  ( 0 . 5 ) :  +  ( 1 . 5 )  2  +  ( 1  .  5  )  2  -  ( 0 . 5  )  2  1 
L 

=  3  x  To.r  +  2.254  2.25  +  0.2  5]=  3  x  5  =  i5 


(c-1)  Between  group  SS  =  15 

(c-2)  Number  of  degree  of  freedom  *  m  =  k  -  1  =  3 


The  analys is-o f-var iance  table  (10.2)  then  tak^s  the  form: 
Source 


" Between 
Cells  " 

"Within 

Cells" 


Sum  of 
Squares 

15 


Degrees 
of  Freedom 

3 


S*_-L2  =  5 


**  -  i  -  1 
8 


^4 


=  j 


Total 


23 


11 


Entering  the  F  table  with  m  3 ,  N  -  8,  and  0.90  (a  significance 
level  of  10%),  the  critical  value  of  F,  F-,  is  found  to  be  2.92. 
Since  F  =  5  >  2.92,  the  nu  1 1  hypothesis  is  rejected  and  treatment 
effects  are  said  to  be  s  igni  f  icar*- .  The  determination  of  the 


numerical  magnitude  of  this  effect 
or  grou’'  means. 


is  made  using  the  treatment 


Here,  c.ie  can  see  that  fo*’r  treatments  are  being  compared  at  the 

same  time. 


3.5  THE  TWO  FACTOR  EXPERIMENT 


The  discussion  of  this  type  of  experiment  is  included  as  an 
tration  of  the  manner  by  which  comp li cat  ions  are  introduced 
Furthermore,  we  can  take  this  opportunity  to  introduce  sane 
of  the  "de  ign  of  experiments'  terminology. 


i  1 1  u  s  - 
more 


This  type  of  design  is,  obviously,  used  when  there  are  two 
important  factors  which  are  suspected  as  'causes  of  failures  i-- 
particular  problem.  Such  combinations  as  humidity  and  umbo  era 
temperature,  pressure  head  and  fluid  density,  voltage  and  age  of 
equipment,  or,  even  time  of  day  and  route.  (This  latter  set  of 
conditions  applies  to  designing  v:n  experiment  undertuxon  to 
determine  the  "effects"  of  departure  time  and  route  used  on  the 
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t imo  to  got  from  work  to  home).  Arbitrarily,  we  shall  call  one 
factor  "treatments"  and  the  second  factor  "environments".  The 
different  "treatments”  shall  be  denoted  oy  capita]  letters,  A,  B, 
etc.,  ana  the  "environments"  by  numerical  subscripts  B1 , Bj ,  ...., 

etc.  Specimens  are  assigned  "at  random"  to  the  different  "treat¬ 
ment-environment"  combinations  and  observations  are  recorded. 

We  shall  assume  that  there  are  an  equal  number,  n,  of  observations 
from  each  "treatment/environment"  combination.  The  letter  "r" 
shall  denote  the  number  of  different  treatments  ana  the  letter 
"s'  shall  denote  the  number  of  environments. 

The  model  is  that 


where 


+  T,  +  b,  t  v>3  +  e; 


is  the  true  overall  moan; 


(2)  t-,  is  the  "main  effect"  of  treatment  a  and 


t  ,  *  0 


■•-here  the  summation  is  over  the  r  different 
treatment s ; 

(3)  b,  is  the  "main  effect  of  the  i*  environment 


4)  all  the  .  represent  sample  values  from  a 
n  ( x ;  d  ,  r )  po  pu  1  a  t  i.  on  ; 

5)  the  \  ,  are  called  the  "interaction  between 

the  "treatments"  and  the  "environments"  and 
s 

r" 

'  v.,  0  for  each  •>  treatment  and 


d  t  r  o  i  c'ck  i  . 
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=  3  x  [(0.5)8  +  (1 . 5) 8  +  ( 1 . 5) 8  +  (0.5)8] 

’  3  x  [0.25  ♦  2.25  +  2.25  +  0.2s]  =  3  x  5  =  15 
(c-1)  Between  group  SS  =  15 

(c-2)  Number  of  degree  of  freedom  =  m  =  k-  l=  3 

The  analysis-of- variance  table  (10.2)  then  takes  the  form: 


Source 


Siam  of  Degrees 

Squares  of  Freedom 


F 


"Between  15 

Cells" 


3 


fSr-  5 


"Within  8 

Cells" 


8 


1 


Total  23  11 

Entering  the  F  table  with  m  =  3,  N  =  8,  and  0.90  (a  significance 
level  of  10%),  the  critical  value  of  F,  F0 ,  is  found  to  be  2.92. 
Since  F  »  5  >  2.92,  the  null  hypothesis  is  rejected  and  treatment 
effects  are  said  to  be  significant.  The  determination  of  the 
numerical  magnitude  of  this  effect  is  made  using  the  treatment 
or  group  means. 


Here,  one  can  see  that  four  treatments  are  being  compared  at  the 
same  time. 


3.5  THE  TWO  FACTOR  EXPERIMENT 

The  discussion  of  this  type  of  experiment  is  included  as  an  illus¬ 
tration  of  the  manner  by  which  complications  are  introduced. 
Furthermore,  we  can  take  this  opportunity  to  introduce  same  more 
of  the  "design  of  experiments"  terminology. 

This  type  of  design  is,  obviously,  used  when  there  are  two 
important  factors  which  are  suspected  as  "causes  of  failures"  in  a 
particular  problem.  Such  combinations  as  humidity  and  ambient 
temperature,  pressure  head  and  fluid  density,  voltage  and  age  of 
equipment,  or,  even  time  of  day  and  route.  (This  latter  set  of 
conditions  applies  to  designing  an  experiment  undertaken  to 
determine  the  "effects"  of  departure  time  and  route  used  on  the 
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These  "interaction"  terms  represent  either  "reinforcement"  or 
"counteraction"  of  the  effect  of  either  the  "treatment"  or  the 
block  by  the  other.  The  graphical  presentations  in  Figure  10-41 
where  Temperature  represents  "environments"  and  Voltage  Levels 
represent  "treatments",  illustrates  different  possible  situations 
that  might  arise. 

MS  ^ 

Again,  X  =  —— — —  is  an  estimate  for  u* 
ns  r 


X*  =  —  ~s~  is  an  estimate  for  n  +  tA: 


X,  =  — is  an  estimate  for  u  +  b,- ; 

»  n  r  1 

n 

^  I  XA1  1  is  an  estimate  for  u  +  tA  +  bx  +  yAi 
j=l 

XA1  -  XA  is  an  estimate  for  bv  +  yAl 
XAl  -  "XA  -  Hi  is  an  estimate  for  Yai  -  M- 

%  -  XA  -  an  estimate  for  yai 

s  u 

The  total  sum  of  squares,  ^  fan  i  ~  ^}a  •  is  now  partitioned 

cr  i=i  j=l 

amongst  the  four  sources : 

(1)  "Between  treatments"  Sum  of  Squares 

X  -  x)a 

a 

(2)  "Between  environments"  Sum  of  Squares 

s 

£  (xt  -  x)a 

i=l 

(3)  "Interaction"  Sum  of  Squares 

s 

I  I 

“  i=i 
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4.1  TECHNIQUES 

Boolean  algebra  is  the  science  of  symbols  and  their  combinations 
used  to  describe  and  represent  mathematical  functions  according 
t^  the  rules  of  logic.  It  was  named  for  an  English  mathematician. 
George  Boole,  who,  more  than  a  century  ago,  translated  the  rules 
of  formal  logic  into  mathematical  terms.  This  science  is  based 
upon  three  fundamental  ideas;  (a)  symbols  are  used  to  represent 
logical  operations,  (b)  these  operations  are  governed  by*  the 
rules  of  logic,  and  (c)  these  rules  are  the  same  as  those  for  an 
algebra  of  the  numbers  0  and  1  (binary  algebra) .  There  ere  many 
forms  of  Boolean  algebras  any  combination  of  propositions,  each 
of  which  is  of  a  binary  nature,  can  be  represented  by  Boolean 
algebra.  This  dicussion  is  designed  to  familiarize  the  reader 
generally  with  the  symbolic  logic  used  in  Boolean  algebra; 
illustrations  are  given  of  basic  logical  operations,  as  well  as 
the  symbols  used  for  representing  these  operations,  and  methods 
of  combining  the  symbols  into  sequences  of  logical  operations. 

Using  Boolean  techniques,  the  technologist  can  analyze  or  synthe¬ 
size  switching  systems  in  any  medium.  The  procedures  apply 
equally  well  to  relays,  switches,  valves,  clutches,  flip-flops, 
transistors,  saturable  reactions,  in  fact,  any  system  or  "ON-OFF" 
or  binary  device  is  amenable  to  this  logic  design  technique. 


There  are  as  yet  no  standard  symbols  for  Boolean  Algebra  functions. 
The  logic  literature  varies  widely  in  this  respect.  This  section 
presents  some  of  the  symbols  and  word  definitions  in  common  use. 

Boolean  facilitates  the  reduction  of  a  problem  to  simplest  form 
for  efficient  processing  by  digital  equipment.  Functions  can  be 
substituted  and  redundancies  eliminated  by  analysis  in  Boolean 
form.  As  in  any  algebra,  it  is  necessary  to  know  the  character¬ 
istics  of  all  terms  before  the  problem  can  be  simplified. 

Anything  capable  of  being  described  can  be  assigned  to  classes. 
Conversely,  classes  can  be  used  for  description.  Anything  can 
be  described  by  the  classes  with  which  it  is  or  is  not  identified. 
The  classes  can  range  from  so  exclusive  as  to  contain  nothing,  to 
so  general  as  to  include  everything.  A  class  containing  nothing 
is  calrled  a  null  class  (usually  identified  by  the  figure  0);  a 
class  including  everything  is  called  an  all-inclusive  class.  All 
classes  between  these  extremes  may  be  identified  by  letters. 

A  total  concept,  usually  identified  by  the  figure  1,  is  the 


10-45 


aggregate  of  all  classes.  The  following  example  will  serve  to 
illustrate  the  relationship  between  the  total  concept,  and  the 
inclusive,  null,  and  intermediate  classes.  Horses  can  be  a 
total  concept.  Horses  can  be  referred  to  as  being  mammals,  as 
being  brown,  or  as  being  in  the  State  of  Maine.  Mammals  would 
be  an  all-inclusive  class  because  it  includes  all  horses.  Brown 
horses  would  be  an  intermediate  class  identified  by  the  letter 
"A".  Horses  in  the  State  of  Maine  would  be  another  restrictive 
class  identified  by  the  letter  "B" .  The  null  class  is  arbitrarily 
designated  as  that  class  which  is  so  restrictive  as  to  contain  no 
horses. 

With  each  concept,  logical  operations  can  be  performed  on  classes. 
The  three  most  common  logical  operations  are  union,  intersection, 
and  complementation.  Table  10.4  shows  several  symbolic  represen¬ 
tations  of  each.  Union  combines  classes  on  an  alternative  basis 
and  is  expressed  as  "or".  For  example,  the  union  of  the  two 
classes  ("A"  and  "B")  cited  above,  would  be  expressed  as  class  A 
"or"  class  B.  This  would  b::  less  restrictive  than  either  class 
because  it  now  admits  all  brown  horses  and,  also,  horses  of  any 
color  in  the  State  of  Maine. 

Intersection  combines  classes  on-a  more  restrictive  basis  and  is 
expressed  as  "and".  For  example,  the  intersection  of  class  A 
and  class  B  would  be  expressed  as  A  "and"  B,  and  all  the  require¬ 
ments  of  both  A  and  B  must  be  satisfied.  Thus  the  intersection 
of  A  and  B  would  specify  brown  horses  in  the  State  of  Maine. 

Complementation  is  the  operation  in  which  items  are  described  by 
signifying  that  they  do  not  belong  to  a  class  or  classes.  For 
example,  horses  that  are  not  in  class  A  would  include  all  horses 
that  are  not  brown.  Horses  that  are  not  in  class  A  and  not  in 
class  B  would  include  all  horses  that  are  not  brown  and  not  in 
the  State  of  Maine.  If  this  example  were  expressed  asxnot  in 
class  A  or  not  in  class  B,  it  would  include  horses  in  either,  or 
none,  of  the  classes,  but  rot  in  both. 

Using  horses  as  an  example  serves  to  acquaint  the  reader  with  the 
logic  of  the  all-inclusive  and  null  classes,  and  the  operations 
of  union,  intersection,  and  complementation.  Figure  10-47  illus¬ 
trates  these  operations  used  in  connection  with  switching  net¬ 
works,  and  includes  symbols  that  replace  the  "ands",  "ors",  and 
"nots".  Referring  to  Figure  10-47,  when  the  concept  is  conduc¬ 
tivity  between  terminals  X  and  Y,  any  condition  which  completes 
the  circuit  between  X  and  Y  is  an  all-inclusive  class.  Because 
we  are  dealing  with  a  binary  algebra,  there  can  only  be  complete 
conduction  or  no  conduction.  With  switch  A  and  switch  B  in. series. 


TYPICAL  BOOLEAN  NOTATIONS 


SWITCHING  CIRCUITRY 


both  A  and  B  must  be  in  the  closed  position  for  the  circuit  to 
be  conductive.  The  circuit  is  not  conductive  if  either  A  or  B  is 
open,  or,  as  expressed  in  Figure  10-47,  A  is  not  closed  or  B  is 
not  closed.  Kith  the  two  switches  in  parallel,  either  A  or  B 
must  be  closed  (both  may  be  closed)  to  make  the  circuit  conduc¬ 
tive;  but  both  A  and  B  must  be  open  (not  closed)  to  make  the 
circuit  not  conductive.  If  the  concept  were  infinite  resistance, 
the  all-inclusive  and  null  classes  would  be  reversed.  Now,  in¬ 
finite  resistance  is  an  all-inclusive  class  and  conductivity  is 
a  null  class.  The  switch  positions  and  descriptions  are  changed 
as  indicated  in  the  figure. 

Table  10.5  indicates  other  logical  operations  and  connectives 
and  a  few  of  the  symbols  commonly  used  to  represent  them. 
"Exclusive  or"  elements  beong  to  one  but  no  more  than  one  of  the 
combined  terms.  "Equivalence"  is  defined  as  that  relationship 
between  two  or  more  sequences  of  operations  whose  resultants  are 
identical.  For  example,  if  AfiB  -  C  and  DflE  -  C,  then  Af)B  is 
equivalent  to  DflE.  implication  is  used  when  one  situation  implies 
another,  such  as  a  football  game  implying  running  or  bodily  con¬ 
tact,  an  electrical  output  from  a  circuit  implying  an  input,  or 
a  current  flow  implying  a  complete  circuit.  Reverse  implication 
is  implication  in  which  the  implied  term  appears  first  for  con¬ 
venience  of  expression;  for  example,  A3B  could  be  written  Bo  A. 
The  symbol  for  inhibition  is  another  method  for  representing  "and 
not"  —  thus  inhibition  can  be  defined  in  terms  of  the  combina¬ 
tion  of  the  previously  described  complementation  and  intersection. 

For  each  logical  function  of  even  the  most  complex  binary  device, 
the  combination  of  inputs  and  resulting  output  can  be  expressed 
as  a  Boolean  equation.  Figure  10-49  illustrates  the  Boolean 
expressions  for  some  basic  logical  functions. 

4.2  CLASSIFICATION  LOGIC 


Various  devices  have  been  developed  which  sort  items  according 
to  the  classes  in  which  they  belong.  In  all  cases  the  device 
asks,  regarding  each  pertinent  class,  "Is  the  element  contained 
in  this  class?"  Because  only  two  possible  answers  exist  (yes  or 
no) ,  a  binary  device  is  capable  of  selecting  all  the  elements  in 
any  class.  The  choice  may  be  made  more  exclusive  by  submitting 
the  selected  elements  to  a  succession  of  inspections  involving 
different  classes,  or  it  may  be  made  more  inclusive  by  including 
elements  from  two  or  more  classes.  The  ordinary  punched-card 
sorter  is  perhaps  the  most  widely  used  automatic  device  employing 
thi?  principle. 
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=  ousuv 


•nt  ire  pepu Lit  i on  f  the  Unite 
pi  sible  t  •'  5 o  1  o c t  the 
iqh  between  I4>>  md  10 i 


■v e p *  ,  It 
Georg i a , 

blue  eyes,  have  criminal  record 
and  own  *■ 
be  aval 

i na  can  he  m  the  torn  of  a 


t  >t 
persons 
c  o.)  Li  n  O  3  f 


ehe  live  in  At  lard 
a  a  v e  b 1  a  n d  e  h air  , 
earn  less  than  $10, 000  per 
Of  course,  the  required  in  formation  mas 
net  properly  recorded,  for  each  person.  The  roc 
n  the  form  of  a  card  for  each  oerson,  containing 


-  n  e  i  r  n on e  s  • 


a  , 

have 
year  , 

t 

r  _ 


spaces  to  represent  up  to  several  hundred  classes, 
ty  include  everyone  who  lives  in  Atlanta,  Gergia; 


di  scr  etc 
oi as s  A  r 

class  B,  everyone  (in  the  United  Stater)  who  weiohs  between  ) 
and  16C  pounds;  class  C,  everyor°  having  blonde  hair;  class  I 
everyone  with  blue  eyes;  class  E,  everyone  with  a  criminal  : 
cord;  class  F,  everyone  who  earns  less  than  $19,000  per  year ; 
etc.  The  fact  that  a  person  belongs  to  any  class  is  indicate 
by  a  hole  punched  through  that  person ’ s  card  in  the  space  re: 
s e n t  i. n q  th at  class. 


A  machine  capable  of  inspecting  one  class  at  a  time  must  b-j 
successively  set  up  to  sort  out  the  cards  which  have  a  hole  in 
the  space  representing  each  pertinent  class.  The  inspection  car. 
be  by  means  of  mechanical  pins,  light  rays,  air  pressure  (player 
pianos  inspect  the  roll  to  determine  which  notes  are  to  be  played), 
electrical  contacts,  etc.  Only  the  selected  cards  need  be  in¬ 
spected  in  s  .cceeding  operations.  Note  that  the  machine  is  per¬ 
forming  the  logical  operation  cf  intersection  ( a"B  C  D’E  EG). 

The  order  of  sorting  is  unimportant  except  that  the  total  number 
of  inspections  can  be  reduced  by  sorting  for  the  most  restrictive 
class  first.  Repetitive  sorting  for  a  single  class  always  re¬ 
sults  in  selection  of  the  same  cards,  so  A  n  A  “A,  or  2a  A.  Tt  is 
evident,  then,  that  numerical  coefficients  have  no  meaning  in 
Boolean  algebra.  Fairly  simple  machines  can  inspec  several 
classes  simultaneously  and  select  only  those  cards  which  fit  all 
the  classes  being  inspected. 


It:  is  usually  more  practical  to  divide  detailed  information  such 
as  weight  ana  income  into  several  subclasses  for  recording.  The 
number  and  range  of  the  classes  are  fixed  according  to  the  detail 
of  the  available  information  and  the  purposes  for  which  it  is  to 
be  used.  Weight  may  be  classed  as  (Bl  )  less  than  100  pounds,. 

(i3.>  )  lOu  to  120  pounds,  (B3  )  120  to  130  pounds,  (B4)  130  to  140 
pounds,  (Bs)  140  to  150  pounds,  ( Bt,  )  150  to  160  pounds,  (B?  )  160 
to  170  pounds,  (Bs )  170  to  180  pounds,  (B0)  180  to  200  pounds, 
and  ( B ; 0 )  over  200  pounds.  Income  may  be  classed  as  (Fj)  0  to 
$2,000,  ( FA  )  $2,000  to  $4,000,  ( F3 )  $4,000  to  $6,000,  (f4)  $6,000 
to  $8,000  ( Fr,  )  $8,000  to  $10,000,  ( Ffi  )  $10,000  to  $15,000,  and 
(F,  )  over  $15,000. 


1  -  '•  I 


The  sorting  machine,  in  order  to  so'  oct 


lt  i  -u.  i  C. 


oust  be 


t  'lose 


either  weight  subclass  B  or  B.-  .  It:  doing  this,  the  r 
performing  the  Boolean  operation  of  union  (B.  B-  )  .  T- 
the  "under  $1 "),  00C  income  class,  the  machine  must  chc 
(1)  all  the  cards  punched  for  income  classes  F  ,  F.  ,  i 


e it her 


1  ,  £ 


F  ,  or  (2)  those  cards  which  are  not  punc.._d  for  class 
not  punched  tor  class  F -  .  The  latter  method  may  be  cci 
because  it  requires  inspection  of  only  two  spaces  i ns t < 
spaces.  When  the  machine  is  set  up  to  select  .arcs  wh : 
not  punched,  it  is  performing  the  logical  operation  of 
mentation.  "Not  F?  and  not  F-  ,  '  is  expressed  ideograp: 
f7'  F-  . 


:es  ir  able 

q  f  ^  j 

a  r  e 
:p  le- 

1 1 1  v  a  s 


The  expression  A  ( B;  .  B-  )  C  D  E'  {F-  '  F~  )  is  the  original  example 
with  the  term  (B=  B_-  }  substituted  for  class  5 ,  and  the  term 
{ F- ' F - )  substituted  for  class  1.  To  avoid  possible  ambiguities, 
the  new  terms  are  isolated  in  parentheses.  The  parentheses, 
like  the  punctuation  marks  in  ordinary  language,  are  used  to 
group  related  terms.  The  logical  rules  of  manipulation  for 
parentheses  in  Boolean  algebra  become  obvious  when  the  expres¬ 
sions  are  converted  to  ordinary  language.  The  expression 
A"  (Be.  B.;  )  C  concisely  states  "A,  and  B6  or  B€  ,  and  C.”  Omission 
of  the  parentheses,  A~Bf  B,,  C,  like  omission  of  punctuation  marks, 
"A  and  Be  or  B«  and  C."  results  in  ambiguity.  Parentheses,  or 
punctuation  marks,  are  necessary  in  oh  is  case  to  prevent  misin¬ 
terpreting  the  expression  as  "A  and  3S  ,  cr  B,.  and  C.". 


The  expression  E'"  ( F6  ~  F-.  )  states  "E,  and  not  F6  and  not  F-,."  Note 
the  the  punctuation  may  be  omitted  without  causing  ambiguity  in 
this  case,  and  E  (F.~'F-)  El  r’-  "  F-,  .  Within  any  Boolean  expression, 
changes  of  signs  from  ~  to  l  or  vice  versa  always  require  the 
use  of  parentheses  to  avoid  ambiguity. 

4.3  RULES  OF  OPERATION 

In  the  example  of  the  card-sorting  machine,  the  function  of 
selecting  the  cards  punched  for  income  classes  F5  ,  F^,  ,  F3  ,  F4  ,  or 
FE  implied  selection  of  the  cards  not  puncheu  for  income  class 
F„  and  not  punched  for  income  class  F, ;  therefore, 


Ft  !iF2HF3UF4;!F„  =*>  F0  r  F7 

and  the  latter,  more  convenient  expression  of  two  terms  was  sub¬ 
stituted  tor  the  five-term  expression.  . this  case,  the  logical 
substitution  of  functions  may  have  been  apparent  without  the  use 
of  Boolean  algebra  but,  in  the  simplification  or  more  complex 
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problems  much  more  maneuvering  is  often 
puiations  which  are  obvious  and  easy  in 
escape  detection  by  direct  analysis . 


necessary  and  many  mani- 
Boolean  form  might 


With  certain  limitations,  which  will  be  mentioned  later,  the 
operation  of  union  is  the  same  as  that  of  addition  in  common 
algebra,  and  the  operation  of  int lor.  is  the  same  as  mat 
of  multiplication.  The  use  of  and  instead  of  +  and  •  does 
not  preclude  the  possibility  of  contracting  the  symbolism  as  is 
done  with  +  and  •  in  such  expressions  as,  for  example,  AB+C. 

The  expression  { A ' B )  C  can  be  contracted  to  AB  C.  Further,  by 
adopting  the  convention  that  the  impli^  cv  nective  is  that 
which  does  not  appear  'n  an  expression  (and  also  that  the  implied 
connective  is  accompanied  by  implied  parentheses)  both  kinds  of 
expressions  can  be  contracted.  Thus:  (A  B  )  VC  can  be  contracted 
to  A3.C;  (A  B)  ~C  can  be  contracted  to  AB'C:  A  (Bk  B-  )  "C  can  be 
contracted  to  A  B-B.-  'C;  and  the  expression 

A"  (B  B,  )  C''Df-E'F7'FT 


can  be  contracted  to  either 

A  B;.  Br  'COd-'E^-f: 


or 


A  (Bgl’BgJC  D  E  F,  F-  . 

Another  method  for  contracting  and  simplifying  Boolean  expressions 
is  to  substitute  one  term  of  an  abviously  true  relationship  for 
its  more  complex  counterpart.  For  example,  this  can  be  done  in 
the  following  ten  relationships  which  are  true  for  all  classes: 

1.  XI X  X;  also  X  X  X.  A  proposition  is  rot  changed  by  repe¬ 
tition,  either  in  an  alternative  or  in  a  restrictive  sense.  For 
this  reason,  coefficients  other  than  0  and  1  have  no  meaning 
(2X  =  X)  . 


2.  0°X  0.  Since  an  "and"  operation  is  restrictive  and  no 

class  can  be  more  restrictive  than  the  null  class,  any  intersec¬ 
ting  combination  which  includes  a  null  class  (0)  must  be  all 
exclus ive. 


3.  X01  X.  All  classes  must  be  entirely  contained 
concept;  therefore  the  added  restriction  of  belonging 
concept  has  no  effect. 


within  the 
to  the 


J 
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4.  y  X.  Vothi  no  Koinnq  to  the  null  class  (0);  hence  the 
inclusion  of  the  null  class  as  an  alternative  is  meaningless. 

5.  X-  1  1 .  Everything  within  the  concept  must  belong  to  any 
unioi.  which  i.. eludes  the  entire  concept  (1)  as  an  alternative. 
No  class  can  include  more  than  the  total  concept. 


6.  XrX  0.  Nothing  can  belong  to  any  class  and  not  belong  to 
that  class. 


7.  XL X:;  1 .  Everything  must  either  beong  to  a  class  or  not 
belong  to  that  class. 


3.  (X)=X.  Double  negatives  cancel  each  other. 


9.  ( XLY ) " X" Y .  Everything  which  does  not  belong  to  either  X 

or  Y  obviously  does  not  belong  to  X  and  does  not  belong  to  Y. 

10.  XoY^ (XrY) .  That  which  either  does  not  belong  tu  X  or  does 
not  belong  to  Y  cannot  belong  to  both  X  and  Y. 


As  an  ordinary  algebra,  there  are  various  manipulations  possible 
in  Boolean  algebra  which  will  permit  simplification  of  expres¬ 
sion.  For  example,  in  ordinary  algebra: 


a3  -  b3  _  (a  b)  (a  -  b) 
a  +  b  a+b  a~b 

In  Boolean  algebra,  manipulations  based  or.  the  following  postu¬ 
lates  are  also  possible. 

11.  Operations  of  union  or  intersection  are  commutative. 


XUY=YUX  (Compares  to  X+Y  =  Y+X  in  common  albegra) 

XHY-YbX  (Compares  to  X*Y  =  Y*X  in  common  algebra, 

The  example  of  the  card-sorting  device  showed  that  the  order  of 
inspection  has  no  logical  significance. 

12.  Operations  of  union  or  intersection  are  associative. 

XU  (YUZ) H  (XIIY)  i  I7=X>JYUZ  (compares  to 

X  +  (Y  +  Z)  =  (X  +Y)+Z=X+Y+Z  in  common  algebra) 


xn  (YOZ)  =  (X  'Y) nz-xnynz  (compares  to 
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X  •  (Y  •  Z)  =  (X  •  Z)  •  Z  -  X  •  Y  •  Z  in  common  algebra' 

Parentheses  are  required  only  when  a  change  cf  connectives  occurs . 

13.  Operations  of  union  and  intersection  are  distributive. 

X!  (Y^Z)  (XhY)'~(X:  Z)  (not  true  for  addition  in  common 
algebra) 

X'' (YUZ)  •  (X^Y) !  (X'  Z)  (compare  to 

X  *  (Y+Z)  =  ( X • Y )  +  (X-Z)  in  algebra) 


Everything  which  is  included  either  in  X  or  m  both  V  and  Z 
must  be  included  in  X  or  Y  and  in  X  or  Z.  Everything  which  is 
induced  in  X  and  either  Y  or  Z  must  be  included  either  in  X 
and  Y,  or  in  X  and  Z. 


In  the  follow-; ng  step  by-step  simplification  of  aUAB  :.o  AUB,  the 
manipulation  performed  is  explained  beside  each  step,  the  circled 
numb  jlS  refer  to  the  rules  of  operation  given  above. 


At  AB-A(BUB)UAB 

sabuaEuab 

^ABUABUABUAB 
-A ( Bl >B)  UABUAB 
=  A  ( Bl  JB )  U B  ( AU A ) 
=  A  (BljB)  UB 
_  -AUB 
A  AB=AUB 


(BUS)  1  ( 7  )  , __A"'  1 "  A  (3) 

Ah  (BUB)  =  AB1  AB  (13) 
ABUABAB  _(l) 

ABUAB-  A(B;iB)  (13) 

ABI  lAB^B  (A!  !A)  (13) 

(AUA)  1  (7),  BH1  B  (3) 

(BUB)  - 1  (7),  (Ad)  A  (3) 
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Cnapter  11 
VERIFICATION 

If  all  people  concerned  in  the  development  of  hardware  for  a 
system  were  omniscient,  there  would  be  no  need  for  a  hardware 
development  phase.  Engineers  could  make  up  specifications  and 
drawings;  the  hardware  would  be  built,  acquired,  and  installed; 
and  the  installed  system  would  work.  Unfortunately,  real  true- 
to-life  human  beings  are  not  omniscient.  They  have  to  acquire 
information.  rphe  basic  purpose  of  experimenting  (testing)  with 
hardware  is  to  provide  information  which  cannot  be  gained  by 
other  means.  Thus,  when  there  is  the  willingness  to  appreciate 
the  need  to  spend  the  money  and  time  to  obtain  desired  infor¬ 
mation  oy  testing,  one  has  the  beginning  of  a  test  program. 

Testing  falls  into  four  general  categories: 

(a)  Development  tests  -  to  get  the  equipment  to  work  and 
evaluate  the  characteristics  cf  its  performance  and  endurance 
under  severe  environments. 

(b)  Qualification  tests  -  to  formally  subject  the  equipment 
to  its  planned  operating  environment  to  display  the  adequacy 
of  the  design. 

(c)  Acceptance  tests  -  to  verify  that  the  production  lot 
falls  within  the  specified  tolerance  range,  ana 

(d)  Demonstration  tests  -  to  provide  an  object ive  evaluation 
of  the  capability  of  the  production  lot  to  continue  to  per¬ 
form  its  function  under  specified  environmental  and  loading 
condit ions . 

Tiiis  course  will  not  dwell  on  tests  and  development  testing  to 
advance  the  state-of-the-art  but  on  the  verification  of  per¬ 
formance  attainment.  The  purpose  of  this  chapter  is  to  discuss 
the  applications  of  testing  to  the  assurance  of  reliability  in 
the  pr  xluct . 

Let  us  discuss,  briefly  the  decision  process.  The  engineer  res¬ 
ponsible  for  a  design  has  the  responsibility  of  providing  an 
equipment  that  will  work  when  required  and  will  continue  to  keep 
working  as  long  as  it  is  needed. 

In  the  process  of  design,  most  of  the  desie  parameters  are 
determined  analytically.  In  developing  his  desiqn,  the  designer 
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makes  r.uroei*  lus  decisi  ns,  based  on  facts  available  to  him,  exper- 
i  i  •nee  a  no  on:;  ineer inq  ru  d  gment  .  The  analytical  method  used  has 
usual ly  been  developed  and  tested  in  numerous  applications .  The 
des  iqner  applies  engineering  judgment  to  the  applicability  of 
that  particular  analysis  to  he  design  he  is  working  on.  r aetc  rs 
influencing  the  ultimate  reliability  of  the  equipment  are  in¬ 
cluded  in  the  choices  he  has  made.  For  exar..pie,  when  he  computes 
the  structural  strength  of  a  foundation,  he  bases  the  strength 
on  achievina  a  low  probability  of  failure  of  that  member  and 
adequate  stiffness  t.  prevent  excessive  deflection  and  control 
unwant  ed  v  ibrat i on . 

Where  the  designer  feels  he  doesn’t  know  enough,  he  orders  tests 
to  team  n  re.  Where  he  makes  a  1  ocision  based  on  extrapolation 
of  previous  exper i once  or  trad i t i >  nal  analysis,  he  orders  a  test 
to  verify  that  the  analysis  is  accurate.  When  lie  is  satisfied 
i  hat  t  lie  dt-s  i>:n  will  meet  the  performance  requirements ,  he  re¬ 
leases  it  to  production. 

After  the  release  of  the  draw’ nos  for  production,  testing  is 
used  t  o  determine  that  the  out  pvt  'f  the  production  line  stays 
within  the  limits  prescribed  by  the  designer.  Tests  are  performed 
to  assure  that  no  gross  dev  i  at  ;  vis  from  sped  f  i  cat  ions  exist. 

And  tests  are  per  formed  to  verify  that  the  equipment  operates 
w ' thin  the  prescribe  :  limits  of  performance  tor  a  reasonable  time. 

Neither  the  engineer  responsible  tor  des inning  a  part,  nor  the 
project  engineer  rospons  ib)  e  for  development,  of  the  system  nor 
the  customer  buying  t  lie  system  can  aarant.ee ,  with  absolute 
certainty,  that  th  '  system  will  work  when  "the  switch  is  closed". 
However  as  tests  are  run  and  data  become  available  each  of  these 
ind  iv i duals  will  qualitatively  assess  this  data  and  will  each 
r --ach  'i  subjective  decision  as  to  whether  or  not  the  probability 
has  been  increased  that  the  system  will  work  when  the  switch  is 
pulled.  Contracts  have,  and  will  be,  cancelled  when  this  assess¬ 
ment  of  quantitative  data  is  negative.  On  the  other  hand,  large 
amounts  of  money  have  been  authorized  and  expended  when  a  specific 
individual  -  with  appropriate  authority  -  has  made  a  positive 
assessment.  As  a  specific  example  of  t h<  latter,  consider  the 
relationship  between  the  success  of  the  nuclear  submarine,  Naut- 
i  lus,  and  the  doc is  ion  to  convert  the  nuclear  submarine,  George 
Washington,  to  a  ioliris  carrier.  The  assessment,  by  the  designer 
or  pro ject  engineer  can  be  a  decisive  factor  in  the  conduct  of  a 
development  program.  It  is,  there  l  re,  necessary  to  provide 
results  of  test  data  in  such  a  form  that  this  qualitative  assess¬ 
ment.  can  be  made  in  a  reasonable  fashion. 
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One  of  the  principal  objectives  of  a  test  program  is  to  provide 
timely  quantitative  data  to  the  engineer  to  enable  him  to  make 
dec  is  ions . 

1  .  ASSURANCE 

The  satisfaction  of  the  judgment  of  the  engineer  that  the  equip¬ 
ment  will  perform  .eliably  will  be  called  assurance.  It  will  be 
based  on  two  factors ,  his  qualitative  satisfaction  that  the  des¬ 
ign  is  "right1'  including  verification  by  testing  that  his  analyses 
were  good,  and  proof  (statistical  verification)  that  the  output 
of  the  production  line  meets  expectation. 

1.1  QUALITATIVE  ASSURANCE 

Part  of  the  evidence  used  by  a  designer  to  satisfy  himself  that 
a  design  will  be  adequate,  is  the  similarity  of  the  design  to 
existing  successful  equipment.  Part  comes  from  his  knowledge 
that  the  analysis  has  been  thorough,  taking  all  important  fact  rs 
into  account.  This  evidence,  when  backed  up  by  tests  that  prove 
to  him  that  his  assumptions  are  substantiated,  is  all  he  needs 
to  give  him  assurance  that  the  design  is  satisfactory 

The  customer,  when  the  design  is  contracted,  needs  a  means  to 
evaluate  this  evidence.  He  vculd  prefer  not  to  bo  faced  with 
major  design  modifications  after  the  equipment  is  built  and 
delivered.  He  would  usually  prefer  that  the  delivery  not  be 
delayed  by  major  correction  of  deficiencies  discovered  during 
acceptance  testing.  To  obtain  assurance  of  this  type  (that 
analyses  were  thorough  and  competent  and  were  supported  by  test 
results)  requires  that  the  customer  somehow  obtain  visibility  of 
the  analyses  and  decisions  of  the  designer  as  well  as  the  results 
of  all  tests  conducted.  This,  in  recent  contracting,  is  done 
through  such  devices  as  reliability  predictions,  design  reviews, 
failure  diagnosis  requirements  in  the  contract.  The  reports 
generated  by  such  activities  provides  the  customer  with  objective 
evidence  that  the  designer  did  a  competent  job.  Equivalent 
evidence  can  be  obtained  that  the  production  and  assembly  areas 
are  competently  controlled  through  enforced  requirements  tor 
inspection.  Ihe  evidence  (data)  furnished  as  the  result  of 
testing,  is  invariably  statistical  in  nature.  No  two  tests  ever 
yield  exactly  the  same  data.  The  random  variations  in  test 
conditions  provide  the  variability  of  data  mentioned  in  our  last 
chapter.  Test  conditions  may  include  a  "bias'.  The  test  of 
every  possible  combination  of  parameters,  to  eliminate  a  bias, 
turns  out  to  require  very  extensive  testing.  Statistical  design 
of  experiments  permits  the  statistician  to  improve  the  efficiency 
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of  testing,  providing  a  test,  program  that  yields  the  most  perti¬ 
nent  information  for  the  least  cost.  This  is  a  specialized  area 
and  always  should  be  performed  by  (or  rather  with  the  assistance 
of)  specialists  in  the  field  of  statistics.  And  like  any  design, 
must  be  done  before  the  tests  are  run. 

1.2  QUANTITATIVE  ASSESSMENT 


It  sometimes  happens  that  an  engineer  who  is  "positive"  that  his 
answer  is  right  turns  out  to  be  wrong.  Qualitative  judgment  can 
never  be  relied  on  entirely.  If  a  system  could  be  operated  for 
an  extremely  long  time,  the  true  reliability  could  be  "measured" 
Lacking  such  extreme  time  of  operation,  the  engineer  would  like 
to  place  a  limit  or  bound  on  what  he  can  say  about  the  system. 
Treating  the  data  for  a  particular  test  as  a  "random"  sample  of 
data  from  the  population  we  might  discuss  what  the  "statistical" 
meaning  of  the  data  is. 

2  .  APPLICATION  OF  STATISTICAL  THEORY 

The  interpretation  of  data  derived  in  a  test  plan  depends,  of 
course,  on  the  purpose  for  which  the  testing  was  performed,  the 
care  with  which  the  tests  were  conSucted  and  documented  and  the 
adequacy  of  the  plan  to  provide  the  information  desired.  When 
the  purpose  of  the  testing  is  to  develop  quantitative  assurance 
that  a  required  reliability  has  been  achieved,  the  quantitative 
measure  of  this  assurance  is  termed  confidence. 

2.1  BASIC  ASSUMPTIONS 


On  developing  equipment  we  start  with  the  assumption  that  every 
part  will  act  in  a  definable  way,  that  every  equipment  produced 
will  be  exactly  alike  ,  exactly  like  the  blueprints.  Every  time 
we  conduct  a  particular  test,  we  should  get  exactly  the  same 
result.  But  we  know  equipment  does  not  come  out  identical.  Each 
equipment  will  be  different,  having  variations  from  the  basic 
design.  Tf  an  infinite  number  of  these  equipments  were  built 
the  minor  variations  of  each  parameter  would  form  some  pattern. 
But  we  cannot  determine  what  this  pattern  is  from  testing  one 
unit.  We  cannot,  usually,  test  all  the  units  built,  particularly 
when  the  test  results  in  destruction.  We  wouldn't  have  any  to 
use.  As  the  nuclear  engineers  have  done  in  a  similar  situation 
we  turn  to  the  field  of  statistics,  making  the  following  assump¬ 
tions  . 

(a)  The  true  reliability  of  the  equipment  is  a  specific 

number,  not  determinable. 
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(b)  Certain  of  the  equipments  will  fail  on  test;  others 
succeed.  The  probabil 1 ty  that  an  equipment  will  succeed  is 
a  measure  of  the  true  reliability  of  tne  system. 

(c)  The  equipments  tested  are  a  sample  drawn  from  a  con¬ 
ceptual  infinite  number  of  equipments  all  built  to  the  design. 

FORM  OF  THE  STATEMENT 

The  numerical  reliability  statement  we  make  about  the  equipment 
as  a  result  of  testing  is  made  in  two  forms:  (a)  Point  estimate, 
and  (b)  Interval  estimate. 


3  .  PEL I ABILITY  ESTIMATION 

3.1  POINT  ESTIMATES 

A  point  estimate  is  a  number  whic..  ;s  an  estimate  of  the  true 
value  of  a  parameter  and  is  based  on  an  available  sample  of 
observed  data.  The  point  estimate  of  reliability  for  example 
is  usually  just  the  ratio  of  the  number  of  successes  to  the 
number  of  trials.  The  point  estimate  of  mean  life  may  be 
computed  as  follows : 


Suppose  that  we  have  conducted  tests  to  failure  of  a  number  of 
essentially  identical  components  and  have  recorded  the  times  to 
failure,  tt  ,  t.-,  ...tB  for  each  failed  component.  The  point 
estimate  of  mean  life  is  the  sum  of  the  ♦'imes  to  failure, 
divided  by  the  number  of  failures,  in  this  case 


Mean  Life 


n 
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3.2  POVjT  ESTIMATION  -  SYSTEMS 

Point  estimates  of  reliability  or  mean  life  can  be  made  for  any 
level  of  assembly,  from  part  to  complete  systems.  Suppose  we 
have  a  system  {^r  subsystem)  made  up  of  three  components. 

Component  A  is  connected  in  series  with  the  parallel  arrangement 
of  c  iponents  B  and  c.  Substituting  the  component  point  esti¬ 
mate  reliabilities  on  A,  3,  and  C  into  the  following  equation 
yields  the  point  estimate  of  the  system  reliability  Rs .  (Refer¬ 
ence  Chapter  4  for  combinations  of  probabilities). 

Rs  ~  +  ”  Rb^C  ) 
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It  should  be  mentioned  that  use  of  the  foregoing  equation  implies 
independence  between  the  constituent  components .  That  is,  there 
is  no  interaction  in  the  sense  that  when  the  components  are  put 
together  in  the  system  an  environment  is  not  created  which  sign¬ 
ificantly  affects  the  reliability  of  any  component. 

3.3  INTERVAL  ESTIMATION 

3.3.1  Confilence:  Interval  estimation  involves  the  construction 
of  a  confidence  interval  on  the  parameter  of  interest,  eg.,  reli¬ 
ability  or  Ml BF .  A  confidence  interval  is  an  interval  which 
covers  the  true  but  unknown  value  of  the  parameter  with  a  given 
degree  of  confidence.  The  construction  of  the  interval  is  based 
on  test  information  and  certain  assumptions  with  regard  to  the 
underlying  distribution.  Hcv  the  interval  is  actually  establish¬ 
ed  will  be  discussed  a  bit  later  in  this  section.  At  this  point, 
we  attempt.  to  provide  some  insight  into  the  confidence  interval 
concept . 

"A  servo-amplifier  has  a  1000  hour  reliability  of  97 %  at  the  90% 
confidence  level.”  What  do  these  words  mean?  The  reliability 
portion  of  the  statement  says:  "There  is  a  97%  probability  that 
the  servo-amplifier  will  function  satisfactorily,  under  specified 
conditions,  for  a  period  of  1000  hours."  The  second  part  adds: 
"There  is  a  90%  chance  that  the  reliability  of  the  servo-amplifier 
is  at  least  as  good  as  we  have  rust  stated.'' 

Many  people  find  this  puzzling.  Why  make  a  statement  about  some 
percentage  of  probability,  then  almost  in  the  same  breath  admit 
that  we  are  not  altogether  certain?  Couldn't  we  wrap  up  the 
percentages  in  a  single  figure? 

To  answer  this  question,  consider  first  the  case  of  a  man  reach¬ 
ing  blindfolded  into  a  bucket  to  pull  out  a  marble.  He  knows 
that  the  bucket  contains  500  marbles,  200  black  ones  and  300 
white  ones.  We'll  say  that  he  counted  the  marbles  himself,  put 
them  into  the  bucket,  and  stirred  them  around  to  assure  random 
selection.  The  man  can  now  say  with  perfe-t  confidence,  "There 
is  a  40%  probability  that  a  marble  withdrawn  at  random  will  be 
a  black  one."  No  confidence  level  needs  to  be  added;  the  state¬ 
ment  just  made  is  known  to  be  100%  true. 

Now  suppose  that  the  man  dips  into  another  bucket,  this  one  con¬ 
taining  a  very  large  number  of  marbles  of  some  unknown  assort¬ 
ment.  We  permit  him  to  withdraw  10  marbles  at  random,  then  look 
at  them  to  obtain  some  idea  as  to  what  may  be  the  composition 
of  the  mixture  in  the  bucket.  Suppose  that,  in  the  sample  of 
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1C,  lies  observes  3  black  marbles,  6  white  ones,  and  one  red  one. 

He  might  then  say,  "There  is  a  30%  probability  that  a  marble 
withdrawn  at  random  from  this  bucket  will  be  black."  But  this 
statement  cannot  be  made  with  complete  assurance  that  it  is 
correct,  for  the  man  doesn't  know  what's  In  uhe  bucket:  he  can 
only  make  an  educated  gues3  based  on  the  limited  amount  of  in¬ 
formation  obtained  from  the  sample.  So  he  has  to  add  a  state¬ 
ment  that  will  indicate  whether  he's  in  a  position  to  make  a 
pretty  accurate  estimate  or  is  only  guessing. 

So  it  is  with  statements  concerning  reliability.  If  we  life- 
tested  all  servo-amplifiers  of  certain  Mark  and  Model  number,  we 
would  have  a  complete  knowledge  of  the  failure  pattern  for  this 
particular  device.  We  could  then  issue,  without  qualification, 
a  statement  of  what  chance  a  single  newly  manufactured  unit 
would  have  of  working  properly  for  1000  hours.  But  nobody  is 
going  to  destroy  an  entire  output  for  the  sake  of  perfect  infor¬ 
mation.  Rather,  we  test  a  few,  in  the  same  fashion  as  the  fellow 
reaching  into  the  bucket  for  10  marbles,  and  make  an  educated 
guess  about  the  characteristics  of  a  product  from  which  our  par¬ 
ticular  sample  was  drawn.  Then,  tc  be  completely  honest,  we  own 
up  that  our  prediction  is  based  on  sample  information  without 
knowledge  of  the  population,  and  attach  a  "percent  confidence 
level,"  so  that  all  may  know  what  test  data  we  had  to  support 
our  estimate. 

Naturally,  the  more  units  we  test,  the  better  can  be  our  guess 
and  the  higher  the  confidence  we  can  have  in  it.  (This  corres¬ 
ponds  to  the  blindfolded  man  being  given  the  opportunity  to 
examine  20  or  50  marbles  from  the  bucket  of  unknown  composition, 
instead  of  only  10  as  mentioned  previously.)  Finally,  the  more 
modest  is  our  reliability  claim  the  higher  can  be  our  confidence 
that  the  reliability  is  at  least  as  high  as  we  are  claiming. 

The  foregoing  discussion  points  up  the  fact  that  we  can  only  make 
statements  about  a  probability  or  reliability  with  perfect  assur¬ 
ance  or  confidence  when  the  sample  observed  is  the  complete  popu¬ 
lation.  Obviously,  there  is  an  intimate  relation  between  confi¬ 
dence  and  probability.  This  relation  is  indicated  in  the  accom¬ 
panying  Figure  11-9.  The  numerical  values  in  the  table  are  com¬ 
puted  by  techniques  to  be  explained  presently.  One  would  guess 
that,  based  on  a  sample  of  10,  the  estimate  of  30%  for  the  pro¬ 
bability  of  drawing  a  black  ie  not  very  good.  As  a  matter  of 
fact,  it  turns  out  that  we  are  only  38.3%  confident  that  the  true 
probability  is  greater  than  30%.  On  the  other  hand,  if  we  had 
observed  12  black  marbles  in  a  sample  of  40,  we  could  be  44.1% 
confident  that  the  true  probability  is  greater  than  30%.  This 
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result  corresponds  to  intuition;  it  is  a  consequence  of  the 
greater  amount  of  data  in  the  second  sample.  Returning  to  the 
sample  of  10  with  3  blacks  observed,  we  would  find  that  the  con¬ 
fidence  in  a  lower  probability,  say  20%,  is  67.8%.  When  we  make 
some  inference  about  probability  (or  reliability)  from  observed 
results,  there  is  some  associated  confidence.  There  is  a  need, 
then,  for  assessing  quantitatively  our  degree  of  confidence  in 
such  an  inference. 

So  far  the  confidence  concept  has  been  discussed  in  a  general 
sort  of  way,  and  some  numerical  results  have  been  given.  How 
are  these  results  obtained,  and  what  precisely  is  the  meaning 
of  a  confidence  interval? 

3.3.2  Binomial  Distribution;  First  let  us  consider  the  binom¬ 
ial  case.  Suppose  we  have  performed  n -independent  trials  and  s 
of  them  were  successful  according  to  some  defined  criterion. 

(In  the  marble  example  n  --  10  and  s~  3.)  Designate  pL  as  the 
lower  bound  at  confidence  level  a;  by  lower  bound  here  we  simply 
mean  the  probability  value  which  corresponds  to  confidence  level 
a »  Using  pIj  as  the  basic  probability  in  the  binomial  probability 
expression,  form  the  probability  of  obtaining  a  result  at  least 
as  good  as  the  actual  observed  one,  and  set  this  equal  to  one  minus 
confidence.  In  mathematical  notation, 

n 

£  (J)  Pi  (1  -  PL)n~  1  -  1  -  a  11.2 

i=l 

For  a  given  value  of  a  this  equation  can  be  solved  for  p^.  Or 
for  a  given  value  ot  pl  the  confidence  a  can  be  determined. 

Having  obtained  p^  for  some  particular  a,  we  now  make  the  state¬ 
ment  that 

p(pL  s  p)  =  or 

which  is  read  as:  "The  probability  that  the  computed  lower  bound 
is  less  than  or  equal  to  the  true  (but  u  nown)  probability  is 
a."  Perhaps  a  better  way  to  state  it,  is:  "The  probability 
(confidence-  is  a  that  the  interval  to  1  includes  the  true 
probability."  Now  what  does  this  mean?  A  computed  interval 
(pL,l)  either  covers  the  true  probability,  or  it  doesn’t  cover  it. 
How  does  this  jibe  with  our  confidence  statement  above? 

To  return  to  the  marble  example,  suppose  we  randomly  draw  re¬ 
peated  (theoretically  an  infinite  number  of)  samples  of  size  10 
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each.  The  number  of  blacks,  of  course,  will  in  general  vary 
from  sample  to  sair.j,1e.  (Previously,  we  had  3  blacks  in  such  a 
sample.)  At  some  fixed  confidence  level,  say  a-  =  0.678,  we  com¬ 
pute  the  lower  confidence  bound  for  each  sample  and  consider  the 
set  of  a  -  confidence  inte  -vals  (pL,i)  thus  obtained.  It  will 
turn  out  that  07.8%  of  the  intervals  will  cover  the  true  proba¬ 
bility  and  32.2%  of  them  will  not. 

Now  we  actually  have  only  one  sample  of  size  10  wherein  3  black 
marbles  were  observed  and  p^  -  0.2  for  a  =  0.678.  If  we  make 
the  claim  that  the  interval  (0.2,1)  contains  the  true  probability, 
we  are  either  right  or  wrong.  However,  since  this  interval  is 
one  of  many  possible  intervals,  67.8%  of  which  cover  the  true 
probability,  our  particular  claim  has  a  67.8%  chance  of  being 
correct.  It  is  only  in  the  sense  of  the  percentage  of  correct 
claims  that  we  says 

P (0 . 20  s  p)  =  0.678 

Solutions  to  equation  11.2  can  be  found  in  the  tables  of  the 
binomial  probability  distribution  (1). 

In  the*  above  description  of  the  meaning  of  "confidence  interval", 
it  can  be  seen  that  the  frequency  interpretation  of  probability 
is  used;  in  particular,  repeated  sampling  from  the  same  den¬ 
sity  function,  a  given  proportion  of  such  intervals  will  cover 
the  true  parameter  value.  Unfortunately,  the  situation  wherein 
there  is  repeated  sampling  is  the  exception  rather  than  the  rule. 
Most  frequently,  we  have  one  sample  and  have  to  base  our  confi¬ 
dence  statement  on  that  one  sample's  data. 

The  type  of  confidence  interval  obtained  from  equation  11.2  or 
from  the  reliability/confidence  charts  Figures  11-29  to  11-34 
1 9  a  one-sided  interval.  It  is  called  one-sided,  because  the 
upper  end  of  the  interval  is  always  unity.  There  is  also  - uch 
a  thing  as  a  two-sided  interval,  whose  upper  limit,  as  well  as 
the  lower  limit,  is  variable  from  sample  to  sample.  The  confi¬ 
dence  concept  of  a  two-sided  interval  is  exactly  the  same  as 
that  of  the  one-sided,  except  in  this  case  we  are  saying  that  the 
confidence  is  the  probability  that  the  interval  --  lower  bound  to 
upper  bound  (where  the  upper  bound  is  less  than  1.0)  --  contains 
the  true  value.  Our  discussion  will  be  limited  to  the  one-sided 
Interval  because  in  reliability  applications  we  are  usually  con¬ 
cerned  more  that  the  reliability  exceeds  a  certain  minimum  than 
that  it  will  be  found  within  an  interval. 

Let  us  consider  briefly  a  simple  application  of  binomial  confi- 
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dence  intervals.  Suppose  in  a  particular  missile  program  20 
shots  have  been  fired  wxth  2  failures.  We  assume  here  a  binomial 
situation,  obviously  an  over-simplified  assumption.  The  missile 
reliability  cannot  be  expected  to  be  the  same  for  each  shot.  Nor 
would  shots  be  independent  of  one  another,  since  fixes  and  improve¬ 
ments  are  constantly  made  in  the  program  as  a  consequence  of  fir¬ 
ing  data.  Recognizing  these  limitations,  we  still  assume  binomial 
and  make  use  of  the  chart  on  Figure  11-3  1.  This  figure  provides 
a  graphical  solution  to  equation  11.2  for  the  case  of  two  failures 
in  any  number  of  tests.  Finding  20  trials  along  the  bottom, 
follow  the  line  up  to  the  95 %  confidence  curve  (the  top  curve). 

The  intersection  falls  between  .71  and  .72  (actually  .716).  This 
gives  us  a  0-716  lower  bound  reliability  at  the  0.95  confidence 
level.  At  the  conclusion  of  the  program  the  results  are  5  fail¬ 
ures  in  50  launches,  giving  the  same  point  estimate  reliability 
of  0.9.  Superficially  there  has  been  no  improvement.  However, 
using  the  chart  on  Figure  11-34,  since  we  now  have  5  failures  in 
50  tests,  we  now  find  that  the  indicated  reliability  is  about  .80 
(actually  .798)  so  we  can  claim  a  reliability  at  least  as  good 
as  0.798  at  0.95  confidence.  Our  improved  reliability  claim  is, 
of  course,  a  consequence  of  the  larger  sample  size.  If  the  pro¬ 
gram  reliability  specification  was  0.75  at  95%  confidence,  it 
was  not  met  after  20  shots.  It  was  met  after  50.  The  point  we 
make  here  is  simply  that  attainment  of  a  rcliability/'conf idence 
specification  depends  upon,  among  other  things,  the  number  of 
tests  specified  in  a  program  and  conversely,  our  requirement  for 
confidence  determines  the  number  of  tests  required.  Therefore, 
such  specifications  should  be  imposed  on  a  program  only  after 
careful  deliberation  and  compromises  between  the  specifications 
and  program  testing  costs. 

3.3.3  Exponential  Distribution:  The  interpretat  ion  of  confi¬ 
dence  intervals  in  terms  of  percentage  of  correctness  of  claims 
is  valid  regardless  of  the  underlying  distribution.  What  does 
depend  upon  the  distribution  is  the  technique  of  computing  the 
interval.  So  far  we  have  talked  in  terms  of  the  binomial  assump¬ 
tions,  one  of  considerable  importance  in  reliability  practice. 

Another  important  distribution,  as  pointed  out  in  chapters  4  and 
5  is  the  negative  exponential  distribution  of  time-to- failure 
corresponding  to  the  underlying  condition  of  constant  failure 
rate.  We  now  consider  the  technique  for  computing  a  one-sided 
confidence  interval  for  this  case. 

The  formula  for  reliability,  it  will  be  recalled,  is 

R  -  e-Xt  # 

where  >  is  the  underlying  constant  failure  rate  and  is  the 
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mission  or  operating  time. 

Since  in  a  practical  situation  the  amount  of  test  data  is  finite, 
we  cannot  determine  X  precisely.  We  are  once  again  confronted 
with  the  need  to  make  some  claim  about  a  value  at  soma  confidence 
level.  Epstein  and  Sobel  (2)  wove  shown  essentially  that  the 
upper  confidence  bound  on  X  at  confidence  or  is  given  by 

.  _  X3  (a,  2r  +  2)  U-3 

2T 

where  r  is  the  number  of  failures  experienced  in  a  test  termin¬ 
ated  at  a  prespecified  total  accumulated  time  T.  The  2r  +  2  in 
formula  11.3  refers  to  the  degrees  of  freedom  of  the  xa  (chi- 
square)  variable.  If  the  test  was  specified  to  terminate  upon 
the  rth  failure,  then  2r  degrees  of  freedom  should  be  used. 

At  this  point  a  few  words  about  the  chi-squ?re  tables  may  be 
appropriate.  The  chi-square  tables  give  the  value  of  a  distri¬ 
bution  quite  useful  in  statistics  (Figure  11-14)  one  of  their 
main  uses  being  the  determination  of  confidence  limits  applicable 
to  the  negative  exponential  distribution.  The  degrees  of  free¬ 
dom  determines  the  shape  of  the  distribution.  Its  only  interest 
to  us  is  its  use  as  a  parameter  to  determine  which  of  the  distri¬ 
butions  fits  the  data  we  have  available.  A  tabulation  of  xa  for 
degrees  of  freedom  from  1  to  30  is  given  in  Figure  11-28.  The 
entries  in  this  table  are  those  values,  x3  for  which  the  left- 
hand  area  under  the  curve  is  equal  to  or.  In  our  application 

\o 

a  =  j  f(\a)  dx*  11.4 

o 

OB 

where  f(\a)  represents  the  \a  density  function  and  I  f(xJ)dx3*l. 

o 

Based  on  equation  11.4  one  may  make  the  confidence  statement 
which  reads  : 

P(X7J  *  X)  -  a  11.5 

For  a  numerical  example  let  us  return  to  the  servo-ampli f ier 
mentioned  at  the  beginning  of  this  discussion  on  interval  esti¬ 
mation.  Let  us  assume  that  either  one  servo  was  tested  to  75,500 
hours  or  that  100  identical  specimens  were  tested,  each  to  755 
hours,  and  that  no  failures  occurred. 

The  information  and  data  are  summarized  as  follows: 


1  1  -  1  5 


Mission  time,  t.  -  1900  hours 

Total  tost  t ’me,  T  7  5, 5 '0  hours 

Failures,  r  o;  degrees  of  freedom  =  2r  +  2  -  2 

Confidence,  a  -  0.90 


Applying  formula  11.3,  as  follows  X  =  xc'/2T.  For  two  degrees  of 
freedom  and  a  =  .90  the  value  given  for  x.~  is  4.61  so 


4.61 

'  u  ~  2  x  7  5,500 


.0305 


We  find  that  at  the  90°/.  confidence  level  the  upper  bound  on  fail¬ 
ure  rate  is  >  „  G.03C5  failure  per  thousand  hours.  Tne  corres¬ 

ponding  90°/-  lower  b~und  on  MTBF  is  9p,  -  1/ty  -  32,790  hours, 
which  means  that  we  are  90%  confident  that  our  servo  has  an  MTBF 
of  32,790  hours  or  more.  For  the  1000  hour  mission  or  operating 
time  the  90%  lower  confidence  bound  on  reliability  is 
-u.0305 

Rl  -  e-  =  0.970,  which  of  course,  is  the  figure  cited 

earlier.  This  result  means  that  we  may  make  the  claim,  with  90% 
assurance,  that  the  probability  of  successful  servo  operation  for 
1000  hour  period  is  0.97  or  better. 


The  servo-amplifier  example  above  was  worked  out  by  direct  com¬ 
putations  from  the  pertinent  formulas  under  the  negative  expon¬ 
ential  assumption.  It  is  possible  to  obtain  essentially  the  same 
an.  wers  by  use  of  the  reliability  test  demonstration  chart, 

Figure  11-29,  This  follows  because,  although  the  charts  were 
derived  under  the  binomial  assumption,  they  approximate  the 
negative  exponential  case  very  well  in  the  regions  where  n  »  r , 
i.o.,  where  the  number  of  test  cycles  is  much  larger  than  the 
number  of  failures.  The  r  for  use  in  the  charts  is  interpreted  as 
the  number  of  mission  cycles  tested,  i.e.,  n  -  T/t.  In  our  servo 
problem  n  7  5.6  >>  0.  Having  used  the  appropriate  chart  to 
obtain  at  some  confidence  a,  w«  can  compute  the  corresponding 
MTBF  and  failure  rate  bounds  by  9^  --  -t/lnRL  and  x  y  -  1/ 9L  “  lnRjyt. 
The  student  may  verify  for  himself  that  this  procedure  gives 
essentially  the  same  answers  as  those  computed  by  the  chi  square 
i  ormu 1  a . 


If  one  desires  to  demonstrate  a  specified  reliability  at  a  stated 
confidence  it  i  •»  obvious  that  the  testing  requirements  can  be 
established.  Figure  11-29  through  11-34,  for  example,  can  be  used 
for  this  purpose.  The  requirements  would  be  in  terms  of  a  given 
number  of  failures  in  a  required  number  of  test  cycles.  If  the 
requirements  are  met  in  actual  testing,  then  the  reliability/ 
confidence  requirement  has  been  met. 

Examination  of  Figure  11-29  to  11-34  will  show-  that  as  reliability 
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requirements  increase,  tests  required  to  demonstrate  that  the 
reliability  has  been  achieved  at  a  stated  confidence  increase 
proportionately.  An  order  of  magnitude  increase  from  .9  to  .99 
say  requires  an  order  of  magnitude  increase  in  testing  from  22  to 
235.  This  is  shown  in  figure  11-17 

This  indicates  that  reliability  testing  to  reasonably  high  con¬ 
fidences  may  cost  excessively,  both  in  terms  of  test  cycles  and/ 
or  time  required,  number  of  specimens,  and  money. 

3.3.4  Summary :  The  use  of  testing  to  develop  the  statistical 
level  of  confidence  that  the  design  has  achieved  a  high  level  of 
reliability  is  limited  by  the  amount  of  money  and  time  the  cus¬ 
tomer  can  afford.  The  need  for  a  chosen  level  of  high  confidence 
depends  on  what  other  proof  is  available  that  will  give  the 
designee  and  customer  "reasonable"  assurance  of  high  reliability. 

as  discussed  earlier,  qualitative  asautenee  suppetti  the  quanti¬ 
tative  aBBUtanoe  teonfcidehee)  derived  from  a  success fully  com¬ 
pleted  demonstration  plan.  It  should  not  supplant  it.  The  de¬ 
gree  of  support  offered  by  qualitative  assurance  must  be  evaluated, 
on  the  basis  of  the  evidence  shown,  by  the  customer.  Based  on  the 
total  evidence  —  his  satisfaction  that  the  designer  has  been 
complete  and  competent  in  his  analysis,  that  the  inspection  on 
the  production  line  is  adequate,  that  the  testing  supports  the 
analysis  and  the  demonstration  tests  yields  a  reasonable  confi¬ 
dence  though  not  able  to  be  specified  quantitatively  that  the 
achieved  reliability  has  been  met  —  the  customer  can  accept  the 
product.  Since  the  evidence  cannot  be  submitted  until  the  acts 
are  performed,  the  customer  must  make  the  initial  presumption 
that  the  various  factors  of  proof  required  will  be  favorable, 
deciding  before  negotiating  the  contract  just  what  evidence  he 
is  willing  to  accept  to  convince  himself,  and  how  much  he  is 
willing  to  pay  for,  then  negotiate  the  contract  and  administer  it 
to  be  sure  he  gets  the  quality  of  design  and  production  desired. 


4.  DEMONSTRATION  TESTING 

As  pointed  out  earlier,  qualitative  assurance  should  support  the 
demonstration  plan,  not  eliminate  it.  We  recognize  that  we  need 
an  acceptance  test  plan  to  decide  whether  or  not  a  requirement 
has  been  met.  The  purpose  of  this  section  is  to  explain  how  to 
develop  a  test  plan  or  how  to  understand  a  test  plan  proposed  by 
a  contractor. 

It  is  generally  recognized  that  a  correct  decision  as  to  whether 
or  not  to  accept  could  be  effectively  guaranteed  if  a  sufficient 
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amount,  of  testing  wi'rp  done.  Usually,  schedule  and  budgetary 
limitations  do  not  permit.  We  are  then  forced  to  make  this  de¬ 
cision  with  a  limited  amount  of  testing,  i.e.,  on  a  sample  the 
natural  inference  is  that  a  correct  decision  cannot  be  100?4 
guaranteed.  This  implies  that  the  contractor  is  takinq  some  risk 
r.hci.t  equipment,  meeting  or  exceeding  the  specification  will  be  re¬ 
jected.  The  customer,  on  the  ocher  hand,  is  taking  some  risk 
that  he  will  be  accepting  sub-standard  equipment.  Statistics 
gives  both  parties  the  capability  of  numerically  assessing  the 
magnitude  of  these  risks  as  well  as  determining  required  changes 
to  the  test  plan  should  be  magnitude  of  these  risks  prove  unde¬ 
sirable  to  either  cv"  both  of  the  parties. 

4.1  TESTING-  STATISTICAL  HYPOTHE SIS 

The  producer  wants  to  be  reasonably  sure  that  equipment  meeting 
the  requirements  is  accepted.  He  wants  to  keep  his  risk  to  as 
small  a  value  as  possible.  He  would  like  to  keep  the  “producer's 
defined  as  the  probability  that  an  equipment  meeting  the  require¬ 
ments  is  rejected,  to  a  numerically  small  fraction.  The  customer 
similarly  wants  to  keep  the  " consumers  risk",  defined  as  the  pro¬ 
bability  that  an  equipment  not  meeting  the  minimum  acceptable  re¬ 
quirement  is  accepted,  to  a  numerically  small  fraction.  We  thus 
need  a  numerical  assessment  of  the  magnitude  of  these  risks. 

4.2  _ BASIS  OF  TEST  PROCEDURES 

The  establishment  of  a  numerical  value  to  the  magnitude  of  the 
risks  involves  the  extent  of  testing.  Consider  for  example: 

An  engineer  has  two  theories  to  explain  a  phenomenon.  In  theory 
1  the  probability  that  the  phenomenon  will  occur  on  any  trial  is 
50%.  Under  theory  2  the  probability  is  only  31%.  He  would  like 
to  learn  which  theory  is  correct.  To  find  out,  he  decides  to 
conduct  an  experiment  10  times  and  record  whether  or  *  ->t  the 
phenomenon  occurs.  In  attempting  to  interpret  the  result  of  the 
experiment  as  to  which  theory  to  prefer  he  feels  that  the  decision 
should  invoice  the  relative  probabilities  of  observing  this  re¬ 
sult.  when  p  =  .50  and  when  p  -  .31.  Setting  up  the  hypothesis 
that  p  -  .50  as  what  is  usually  called  the  null  hypothesis.  Ho, 
he  can  test,  this  against  the  alternate  hypothesis,  H]_,  that  P-.31 
Using  a  tabular  form.  Figure  11-19  he  computes  the  probability 
of  observing  each  of  the  '’'•'ssi.ble  outcome'’  on  each  hypothesis. 

Since  the  events  are  not  time  dependent,  but  rather  independent 
trials,  the  binomial  equation  will  be  used 


risk" 
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In  this  table  he  notes  that  10  successes  is  119  times  more  likely 
under  the  hypothesis  p  -  J./2  than  it  is  under  the  hypothesis 
p  =  .31.  If  10  successes  occur,  therefore,  the  hypothesis  p  =  1/2, 
is  to  be  preferred.  In  a  similar  way  0  successes  is  (1/2 5th)  as 
likely  under  p  -  1/2  than  under  p  -  0.31.  If  no  successes  occur, 
the  hypothesis,  p  =  .31,  is  preferred.  It  is  intuitively  clear 
to  this  engineer  that  his  decision  should  be  based  cn  a  ratio  of 
probabilities  under  the  two  hypotheses  of  the  observed  result. 
However,  the  observance  of  a  particular  result,  even  though  highly 
favorable  to  one  hypothesis,  does  not  negate  Lhe  possibility  that 
the  other  hypothesis  is  true.  Therefore,  if  this  engineer  is  to 
reach  some  conclusion  and  then  to  take  action  on  the  basis  of  tlm's 
conclusion,  he  must  be  willing  to  accept  some  risk  of  reaching  an 
erroneous  conclusion. 

Obviously  he  will  reject  the  hypothesis  p  =  .50  and  prefer  p  =  .31 
if  two  or  less  trials  result  in  success,  this  gives  him  5 ; 1  odds 
of  being  right.  If  p  =  .50  is  actually  the  correct  solution,  the 
probability  of  achieving  0,  1  or  2  successes  is  Lhe  sum  of  th~ 
probabilities  for  these  numbers  of  successes  (Figure  11-20  )  or 
about  .055.  This  is  the  risk  he  takes  of  being  wrong,  of  coming 
tc  an  erroneous  conclusion  based  on  the  tests.  This  risk  is 
usually  called  the  producers  risk,  the  probability  that  an  obser¬ 
vation  will  fall  into  a  region  of  rejection  when  the  hypothesis 
is  in  fact  true. 

Alternately,  if  p  is  in  fact  .31  the  probability  that  he  will 
observe  the  result,  0,  1  or  2  successes,  is  .356,  leaving  a  pro¬ 
bability  of  .644  that  some  other  result  will  be  obtained  (3  or 
more  successes).  This  value  .644  is  the  probability  that  he  will 
accept  the  hypothesis  p  =  .50  when  p  is  in  fact  .31.  This  is  the 
consumers  risk,  the  probability  of  incorrectly  accepting  the  null 
hypothesis  when  the  alternate  hypothesis  is  true. 

Figure  11-20  shows  that  this  engineer  does  have  the  capability  of 
controlling  the  magnitude  of  the  risk  by  appropriate  selection  of 
the  critical  region. 

Of  course,  the  engineer  still  has  a  remaining  decision  --  that 
of  whether  or  not  the  risks  are  acceptable.  If  he  does  find  a 
region  of  rejection,  e.g.,  0  or  1  or  2  successes  that  has  accep¬ 
table  values  of  producers  and  consumers  risks  the  derivation  of 
the  test  procedure  is  complete. 


His  choice  of  decision  criteria  (two  or  less  occurrences  in  ten 
trials)  still  leaves  him  a  risk  of  64%  that  he  will  mcorectly 
accept  the  hypothesis  p  =  .5.  Changing  the  acceptance  region  to 
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0,  1,  2  or  3  occurrences,  reduces  his  (the  consumer's)  risk  to 
38%  but  increas.-5  the  producers  risk  to  17%. 

If  the  engineer  cannot  find  a  region  of  acceptance  that  provides 
acceptable  values  of  the  risks,  he  must  consider  the  use  of  more 
than  10  trials. 


4.3  FORMAL  TEST  PROCEDURES 

Using  the  concept  ^ust  introduced  we  can  develop  a  formal  test 
procedure  to  determine  the  rejection  region  for  a  reliability 
demonstration  test. 


A  new  system  is  assumed  to  have  a  negative  exponential  density 
function  for  times  to  failure.  The  required  reliability  is  425 
hc’^is,  mTBF.  A  test  is  to  be  designed  to  test  this  value  (425 
hrs .  MTBF)  against  an  alternate  hypothesis  of  250  hrs .  MTBF. 
(considered  a  minimum  acceptable  reliability) 

Hq  -  425  hrs 

H,  =  250  hrs 

to  determine  the  form  of  the  region  in  which  the  null  hypothesis 
will  be  rejected,  we  compute  the  ratio  of  probabilities  of  the 
observed  result  under  the  two  hypothesis,  establishing  the  re¬ 
jection  region  such  that  if  the  ratio  Pq/P^  is  less  than  some 
value,  the  null  hypothesis  will  be  rejected.  A  number,  T,  repre¬ 
senting  the  length  of  time  the  test  is  to  b*_  run,  is  computed  such 
that,  if  the  equipment  fails  within  the  test  time,  the  hypothesis 
is  to  be  rejected.  To  determine  the  value  of  T,  assume  Hq  is  true. 
Select  a  value  for  the  producers  risk. 

The  producers  risk  is  the  probability  that  a  failure  will  occur 
prior  to  the  end  of  the  test  (time  -  T)  when  the  MTBF  is  equal 
to  or  greater  than  425  hrs.  For  this  example,  we  wild  select  a 
value  of  0.10  for  the  producers  risk. 


As  you  will  recall  from  our  discussion  oi  confidence,  the 
probability  that  an  interval  contains  the  true  value  oi  MTBF  was 
expressed  by  the  equation 

'  h  2T 


Solving  for  our  test  time,  T  we  find 


xjug 
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T 
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Using  2  degrees  of  freedom,  since-  t'n  test  is  specified  to  term¬ 
inate  on  the  first  failure,  md  .  l-i  probability  that  the  interval 
contains  the  true  value  wo  find 

-  .2  11 

.211  x  425  „ , 

T  =  - - -  44.6  hours 

The  test  procedure  is  to  put  the  system  on  test  for  44.6  hours. 

If  it  fails  before  this  time  Hq  is  rejected.  If  it  does  not  fail 
the  system  is  accepted. 

The  consumers  risk  is  computed  as  the  probability  that  the  system 
does  not  fail  prior  to  44.6  hours  when  the  MTBF  is  really  230 
hours 


p(s  ) 


-  44.6/250  -.18 

e  -  o 


.8  3  5 


This  means  that  the  test  procedure  will  accept  systems  with  an  MTBF 
of  2  50  hours  8  3.5%  of  the  time-. 

Systems  with  MTBF  lower  than  250  hours  (considered  unacceptable) 
will  be  accepted  with  gradually  reducing  probabilities.  The 
MTBF  of  250  hours  is  a  limiting  condition  defining  the  consumers 
risk  as  a  probability  approaching  83.5  that  bad  systems,  systems 
with  MTBFs  lower  than  250  hours  will  be  accepted. 


4.4  SEUUENVIAL  TEST  FLANS 

This  value  of  the  consumers  risk  is  considered  unacceptable.  One 
procedure,  used  in  industry  is  to  superimpose  a  second  test  pro¬ 
cedure  on  top  of  the  first  one.  In  this  second  test  procedure  the 
null  hypothesis  (Hq)  is  the  minimum  acceptable  value  of  MTBF 
(250  hrs).  This  value  is  to  be  tested  against  a  (new)  alternate 
hypothesis,  of  425  hours.  A  test  time  is  to  be  computed  such 
that  a  system  with  MTBFs  equal  to  ol  less  than  250  hours  will 
not  be  accepted  more  than  10%  (.  the  time.  (not»  the  reversal  of 
null  and  alternate  hypotheses.) 


Entering  a  table  of  \*  with  2  degrees  of  freedom  and  the  proba¬ 
bility  of  .90,  since  this  time  we  want  a  probability  of  90%  that 
the  true  MTBF  is  greater  than  250  hours  (10%  risk  chat  it  is  not). 
The  value  of  x'?  is  found  to  be  4.61 


T  = 


x?e 


4.61  x  250 

2 


575.6  Hrv'rs 
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The  system  is  placed  on  test.  If  it  fails  prior  to  the  44.6  hours 
it  is  rejected.  If  it  operates  successfully  for  575.6  hours  it  is 
accepted.  If  a  failure  occurs  between  44.6  and  575.6  hours,  the 
trial  is  continued.  We  are  now  concerned  about  a  second  failure. 

To  compute  the  times  to  the  second  failure,  select  values  from 
the  x3  table  for  the  same  risks  as  before  but  with  4  degrees  of 
freedom  (2  failures) . 

H0  =  425  Hq  =  250 

X2 ( . 10 , 4)  -  1.06  xP  (.90,4)  -  7.78 

Ta  -  22  5  Tj  =  97  0 

If  the  second  failure  occurs  prior  to  225  hours  the  system  is  re¬ 
jected.  If  the  second  failure  has  not  occurred  by  970  hours,  the 
system  is  accepted.  Again  should  the  second  failure  occur  in  the 
interval  225  to  970  houxS ,  no  decision  is  reached.  The  procedure 
is  repeated,  computing  the  times  for  three  failures,  and  so  on. 

A  more  detailed  discussion  of  the  sequential  test  plans  will  be 
found  in  chapter  10. 

The  test  plan  just  described  has  established  the  risks  of  "good" 
(MTBF  >425  hours)  systems  being  rejected  and  "bad  systems  (MTBF  < 
250  hours)  accepted  at  ten  percent.  Suppose,  however  we  redefine 
a  bad  system  as  one  having  an  MTBF  less  than  400  hours.  The 
initial  values  for  the  first  plan  based  on  Hq  =  425  remains  the 
same,  (assuming  the  risks  are  to  be  kept  at  ten  percent) »  A 
comparison  of  this  revised  plan  with  the  previous  one  is  given  in 
Figure  11-25  . 

Test  time  of  921  hours  is  required  if  no  failure  occurs.  If  the 
first  failure  does  occur  in  the  n^-decis ion-  range,  (44.6,  921) 
and  this  i3  quite  likely,  then,  after  repair,  the  system  is  con¬ 
tinued  on  test,  being  accepted  if  the  test  runs  1552  hours  without 
a  second  failure.  The  bureau  might  not  wish  to  buy  a  "used"  com¬ 
puter  with  an  MTBF  of  425  hours  after  being  on  trial  for  1552  hrs . 
The  requirement  of  testing  9  =  425  against  9  -  400  at  10%  risks 
is  not  a  reasonable  requirement  in  that  the  trial  times  involved 
are,  for  all  practical  purposes,  equivalent  to  destructive  testing. 
To  reduce  trial  time,  a  "bad"  computer  may  be  redefined  as  9  =  250 
and/or  the  10%  consumers  risk  may  be  increased. 

Jt  may  be  noted  that  if  you  test  Hq :  9  ®  425  against  :  9  =  400 
at  ct  -  0.10,  the  same  rejection  region,  (0,44.6)  is  obtained. 
Indeed,  the  interval,  (0,44.6),  is  suitable  to  test  9  =  425 
against  any  9  less  than  425  as  long  as  ar=  0.10.  In  this  situation, 
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one  has  been  able  to  derive  one  rejection  interval  to  test  the 
simple  null  hypothesis  9  =  425  against  the  composite  alternative, 
a  *  42  5.  Using  this  common  rejection  region,  one  could  compute 

the  consumers  risk  with  this  test  for  say  a  -  424,  a  =  400,  9  -  250 
etc.,  making  his  choice  on  the  risk  he  was  w'illing  to  take. 

5.  SEQUENTIAL  SAMPLING  PLANS 


5.1  AGREE  PLAN 


Sequential  test  plans  have  been  computed  and  are  available  for 
use  (3)  in  the  report  of  the  Ad  Hoc  Group  on  Reliability  of 
Electronic  Equipment  (AGREE,  Task  Group  3).  This  acceptance 
table  makes  the  following  assumptions: 

(a)  Both  Producers  Risk  and  Consumers  Risk  are  set  at  10  k: 

(b)  The  alternative  hypothesis  has  been  set  at  3  the 

value  of  the  null  hypothesis. 

This  second  assumption  controls  the  Consumers  Risk.  In  effect  it 
is  say inn  the  probability  is  10k  that  an  equipment  whose  MTBF  is 
only  2  3  the  specified  value  will  be  accepted.  If  the  specified 
value  of  MTBF  is  1000  hours,  the  equipment  will  be  accepted  with 
an  MTBF  lower  than  670  hours  ten  percent  of  the  time. 

5.2  ADDITIONAL  SOURCES 

The  AGREE  sequential  test  plan  is  incorporated  as  the  test  plan 
for  Electronics  Equipment  in  MIL  R  22732  (SHIPS).  MIL  STD  105 
provides  plans  for  sampling  by  attributes  and  may  be  used  as 
outlined  in  that  chapter  for  developing  test  plans,  or  for  in¬ 
terpreting  the  meaning  in  terms  of  risk  in  plans  proposed  by 
contractors . 


6 .  REFERENCES 

1.  Tables  of  Binomial  Probability  Distribution  Function, 
National  Bureau  of  standards  Applied  Mathematics  Series  6, 
U.  S.  Government  Printing  Office,  Washington,  !'.  C. 

2.  Life  Testing,  Epstein  and  Sobel,  Journal  of  the  American 
Statistical  Association,  Vol.  48,  1953. 

3.  Reliability  of  Military  Electronic  Equipment,  Report  by 
Advisory  Group  on  Reliability  of  Electronic  Equipment, 


11-2  1 


4  June  1957,  U.  S.  Government  Printing  Office,  Washington, 

D.  C. 

4.  Introduction  to  the  Theory  of  Statistics,  A.  t'.  Mood,  McGraw- 
Hill  Bo'  k  Company,  1950. 

5.  Mathematical  Methods  of  Statistics,  H.  Cramer;  Princeton 
University  Press. 


CHI-SQUARE  (x  )  DISTRIBUTION  TABLE 


11-28 


r-  c 


-5-2 


UJi 


iAeffreei*  of  freedom  (2r> 


ac  «  r-  M  <C  -  irt  ji  r 


*  n  iC'  ai  n  ic  c  e 


ffl  f!  <  C  c*.  <C  C  «“!  <C 


■e  '■£  m  n  ifl  h 


<rt  w>  «r  i  r 

«  n  n  t*  <*■ 


c  —  <n  n  ■*■ 
:-j  N  .N  N 


c*  m  c4  .n  fi 


*'  V  r-  r  x  ^  — 


w  —  r«  o*  r"  n  ■*  »-*  «n  n?  r-  — 


♦  e  tf  n  *  ^  >  *1 

•«e  wi  *■  <•  3* 


-  <J  «  —  r-.  «  C  t 

f*  «rt  O  «r  —  *0  f*  «  «  O  C  (H  >-*> 


m  <  *  ♦  «  <  C  r-  r- 

—■  r-  #c-  t-  <r  c-  «* 


15  ?  3  S  #  •' 


M  r«  ••»  *r  ■+  v5  <•- 


•  «  O  *  *  n  —  c-  — 

*  :  s  " 

—  r? 


*“«•"»*  o  *  r-  *  $> 


.;  S  «  3  a  X  3  £  K  5  %  s 

|JS!  u»op»»jj  jc 


RELIABILITY  TEST  REQUIREMENTS 

ZERO  FAILURES 


RELIABILITY  TEST  REQUIREMENTS 

THREE  FAILURES 


UMBER  UF  1  ESTS 


RELIABILITY  TEST  REQUIREMENTS 

FOUR  FAILURES 


NUMBER  OF  TESTS 


RELIABILITY  TEST  REQUIREMENTS 


12-1 


Chapter  12 

FAILURE  MODES  &  EFFECTS  ANALYSIS 

Page 

1.  USES  OF  FAILURE  MODES  &  EFFECTS  ANALYSIS  12-  3 

1.1  Application  to  Reliability  Prediction  12-  3 

1.2  Application  to  Maintainability  Prediction  12-  4 

1.3  Application  to  Safety  Analysis  12-  6 

1.4  Time  of  Analysis  12-  6 

2.  IDENTIFICATION  OF  CRITICAL  ITEMS  12-10 

2.1  Failure  Effect  Analysis  12-10 

2.2  Reliability  Model  Indexing  Numbers  12-11 

2.3  Criticality  Ranking  12-11 

2.4  Critical  Items  List  12-13 

2.5  Applications  of  Criticality  Ranking  12-14 

3.  REDUCING  EFFECTS  OF  FAILURE  12-16 

4.  SUMMARY  12-17 

5.  REFERENCES  12-18 


i 


12-2 


Chapter  12 

FAILURE  MODES  &  EFFECTS  ANALYSIS 

A  Failure  Modes  and  Effects  analysis  is  a  qualitative  means  of 
evaluating  the  reliability,  maintainability  and  safety  of  a  de¬ 
sign  by  considering  potential  failures  and  the  resulting  effects 
on  a  system.  Basically  the  analysis  involves  the  identification 
and  tabulation  of  the  ways  (or  modes)  in  vh  ich  a  part,  component 
or  system  can  fail,  as  for  example  (1),  a  ball  bearing  may  fail 
from  normal  wearout  or  abnormal  wearout,  or  brinelling.  The 
effect  of  each  mode  is  identified,  as  abnormal  wearout  will  cause 
.increased  noise  and  vibration,  with  rapid  wearing  of  bearing 
parts  and  eventual  destruction  of  bearing  and  seizing  of  the  pump. 

In  using  the  analysis  the  identified  effect  may  be  different  de¬ 
pending  on  the  purpose  for  which  the  analysis  is  to  be  used.  In 
reliability  analysis  the  effect  considered  is  the  effect  on  the 
performance  of  system  function.  In  maintainability  analysis,  the 
effects  include  the  symptoms  by  which  a  failure  could  be  identi¬ 
fied  (as  temperature  of  the  oearing)  and  the  additional  parts 
needing  replacement  due  to  damage  because  of  the  failure  of  the 
part.  In  Safety  analysis,  the  additional  effects  considered 
would  be  damage  to  adjacent  equipment  and  possible  danger  to 
personnel . 

Failure  mode  and  effect  analysis  is  a  systematic  procedure  for 
determining  the  basic  causes  of  failure  and  defining  actions  to 
minimize  their  effects.  It  may  be  applied  at  any  level  of 
assembly  (from  complete  weapons  systems  to  parts).  In  each  case 
the  mode  i-s  described  as  the  way  in  which  the  unit  fails  to  per¬ 
form  its  function.  For  a  missile  system  the  function  of  hitting 
a  target  may  not  be  performed  due  to  guidance  error  or  incorrect 
velocity  due  to  early  engine  shutdown,  etc.  For  a  pump,  failure 
to  produce  the  proper  volume  and  pressure  of  fluid  may  be  due  to 
loss  of  suction  or  bearing  seizure.  In  chapter  3  we  described 
the  breakdown  of  functional  requirements  step  by  step,  identify¬ 
ing  functions  with  hardware  that  performed  the  function.  In  the 
same  way,  in  Failure  Modes  and  Effects  analysis  we  establish  the 
functions  that  the  equipment  is  intended  to  perform  describing  as 
modes  of  failure  ways  in  which  the  equipment  can  fail  to  perform 
the  function.  In  reliability  analysis,  the  effects  are  the  in¬ 
verse  of  the  defined  function  that  is  the  failure  mode  effect 
is  the  failure  to  perform  the  required  function. 

The  analysis  is  performed  to  isolate  and  identify  weaknesses  in 
the  design.  The  final  step  in  the  analysis  is  the  determination 
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of  ways  to  eliminate  or  reduce  th“  probability  of  incidence  of 
critical  failure  modes  to  improve  the  design.  Since  funds  and 
time  are  never  unlimited,  corrective  action  involves  the  assign¬ 
ment  of  priorities  of  effort  based  on  relative  seriousness  of 
the  consequences  (effects)  of  failures. 

1  *  USES  OF  FAILURE  MODES  P  EFFECTS  ANAI-YSIS 

1.1  APPLICATION  TO  RELIABILITY  PREDICTION 

A  method  used  in  predicting  the  reliability  of  mechanical  systems 
is  similar  to  the  method  used  in  predicting  the  reliability  of 
electronic  systems.  A  reliability  block  diagram,  which  is  a 
pictorial  representation  of  a  failure  effects  analysis,  is  a 
basic  part  of  each  method.  In  electronic  systems,  the  blocks 
are  identified  as  parts  or  components.  Failure  modes  or  mechan¬ 
isms  are  seldom  referred  to.  In  mechanical  systems,  however, 
the  blocks  are  identified  by  modes  of  failure,  for  each  part  or 
component.  In  mechanical  system  reliability  predictions,  refer¬ 
ence  is  mad'-'  to  "types  of  failures  of  parts  in  specific  applica¬ 
tion"  rather  than  "parts  failure  rates." 

It  is  evident  that  an  accurate,  precise  definition  of  failure  is 
necessary.  The  definition  of  component  failure  is  as  needed  as 
the  definition  of  system  failure,  in  particular  where  the  compon¬ 
ents  are  those  parts  of  the  system  to  be  use  i  in  the  system  pre¬ 
diction  and  failure  rate  data  is  available  for  them.  The  control¬ 
ling  factor  in  determining  the  meaning  of  component  failure  is 
the  tolerance  of  the  system  to  component  variation  and/or  inop¬ 
erability.  This  tolerance  varies  with  the  type  and  timing  of 
component  performance  variation,  o.q.,  a  sticking  valve  may  or 
may  not  affect  system  performance,  depending  on  whether  the 
valve  sticks  open  or  closed  and  when  the  sticking  occurs.  There¬ 
fore,  component  failures  in  mechanical  systems  often  cannot  be 
defined  except  in  reference  to  that  system.  A  brief  outline  of 
a  method  to  be  used  in  predicting  the  reliability  of  a  large 
mechanical  system  is  as  follows: 

Step  1.  Divide  the  system  into  a  number  of  subsystems  which  can 
be  more  e.vsily  dealt  with.  As  this  prediction  method  involves 
predicting  the  reliability  of  each  subsystem  and  then  i  eccmbining 
these  predictions  to  arrive  at  the  overall  system  reliability, 
the  division  must  take  place  on  a  functional  basis.  Careful  and 
precise  system  and  subsystem  definition  (chapter  4)  is  a  necessary 
prerequisite.  The  block  diagram-  is  useful  in  coordinating  and 
recording  the  functional  breakdown.  Numbers  are  usually  assigned 
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to  the  blocks  for  ease  of  cross  reference.  System  definition 
should  include  time  line  analysis,  environments,  and  definition 
of  failure  at  each  block  level. 

Step  2 .  Make  a  detailed  study  of  the  schematic  engineering  draw¬ 
ings  for  each  subsystem  in  order  to  determine  all  of  the  signi¬ 
ficant  modes  of  failure.  Knowledge  of  the  effect  of  component 
failure  as  well  as  the  subsystem  and  system  reaction  to  failure 
of  the  component  is  necessary.  Definition  of  failure  is  an 
essential  portion  of  the  analysis,  but  it  cannot  be  treated  in 
general  terms;  i.e.,  failure  means  operation  not  in  conformity 
with  some  well-stated  performance  requirements. 

Step  3.  Determine  all  of  the  component  failure  mechanisms  which 
could  lead  to  each  of  the  failure  modes.  Failure  mechanisms  are 
the  basic  physical  causes  of  failure  and  failure  modes  are  the 
reactions  to  failure  mechanisms.  Failure  modes  can  result  from 
the  occurrence  of  any  one  of  a  set  of  failure  mechanisms  or  from 
the  simultaneous  occurrence  of  two  or  more  particular  failure 
mechanisms . 

Step  4.  Make  a  summary  of  all  the  reliability  information  obtained 
and  analyzed  from  the  design  schematic  drawings.  This  is  accom¬ 
plished  by  tabulating  all  of  the  failure  modes  and  making  an 
analysis  to  demonstrate  the  relationships  between  component  and 
system  malfunctions.  (Figure  12-5). 

Step  5.  Using  the  information  compiled  above,  prepare  a  relia¬ 
bility  model  in  the  same  manner  as  Ln  chapter  5. 

6.  Determine  the  probabilities  of  occurrence  of  the  failure 
modes  to  be  used  as  numerical  inputs.  This  type  of  data  may  be 
obtained  from  manufacturers  or  may  be  estimated  from  the  prior 
experience  of  the  engineer.  While  in  most  cases,  the  values 
of  failure  rates  are  approximate,  this  computation  has  great 
power  in  comparing  alternatives.  Reference  (2)  is  an  excellent 
example  of  such  a  computation. 

Step  7  .  Generate  the  system  reliability  prediction  utilizing 
the  reliability  model  and  the  probabilities  associated  with  the 
occurrence  of  each  failure  mode  to  arrive  at  a  numerical  valje 
representing  the  overall  reliability  of  the  system  under  investi¬ 
gation. 

1.2  APPLICATION  TO  MAINTAINABILITY  PREDICTION 


Aa  mentioned  in  chapter  8  the  prediction  of  a  Mean  Time  to  Restore 
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(MTTR)  requires  fir?  :..  tux'  i  dent  1 1 :  eat.  i  on  of  the  parts  subject  to 
failure  and  an  estimate  cl  the  p-v  bable  frequency  of  such  failures. 
The  failure  Modes  and  Effects  .-ma  lysis  requires  the  creati  n  of 
just  such  a  list.  The  documentation  provides  the  necessary  design 
discipline  for  methodically  evaluating  the  probability  of  failure 
and  the  results  thereof  for  t*ade-otf  between  reliability  and 
maintainability  to  achieve  ti."*  ..’'stem  availability  requirements. 

The  failure  modes  approach  refines  the  prediction  reliability 
and  maintainability  to  a  consideration  of  the  various  mechanisms 
of  failures  that  may  be  operable. 

Figure  12-7  provides  an  example  of  a  Failure  modes  and  effects 
analysis  for  maintainability  evaluation  (1).  The  equipment  is  a 
steam  turbo-pump.  Figure  12-3  continues  the  analysis  of  tasks 
to  the  individual  task  elements  as  outlined  in  chapter  8. 

1.3  APPLICATION  TO  SAFETY  ANA  I  ho  IS 

The  safety  aspects  of  equipment  failure  are  investigated  by  a 
Safety  analysis.  Safety  analysis  is  not  restricted  to  human 
safety,  but  includes  the  effect  on  the  total  system,  associated 
or  adjacent,  equipment  and  personnel  in  the  vicinity  either  assoc¬ 
iated  with  the  system  cr  casual.  Starting  from  the  identifica¬ 
tion  of  the  expected  failure  modes,  the  effect  on  the  adjacent 
and  associated  equipment  is  evaluated.  An  example  is  given  in 
Figure  12-9. 


1.4  TIME  OF  ANALYSIS 

Failure  modes  and  effects  analysis  starts  from  the  top  down. 
System  functions  and  failure  modes  are  first  considered  in 
abstretior.,  then  expanded  down  to  the  subsystem,  component  and 

part  level. 

It  is  initiated  during  the  concept  phase  of  a  design,  then  as 
the  design  becom.es  more  clear  ly  defined,  is  expanded  concurrently 
with  the  design.  The  effectiveness  of  the  analysis  in  system 
tradeoffs  is  made  possible  by  its  availability  at  the  time  design 
decisions  are  required.  The  analysis  documentation  must  be  kept 
dynamic  and  current  v  rtn  the  design  clear  through  the  final  test 
and  delivery  of  the  equipment.  It  must  be  available  for  use  as 
design  changes  are  proposed  to  assure  that  the  discipline  pro¬ 
vided  keeps  control  of  the  effects  of  changes  in  reliability  and 
maintainabi 1 ity . 

In  the  failure  effect  analysis  of  the  structure,  no  written 
analysis  accompanies  the  reliability  mode.  During  the  design 
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the  structure  undergoes  an  analysis  involving  design  and  stress 
calculations,  which  can  be  classified  as  a  single  failure  effect 
analysis*  On  the  basis  of  this  analysis,  the  structure  is  strength¬ 
ened/redesigned  at  those  points  where  possible  failure  will  occur. 
For  this  reason,  it  can  be  stated  that  the  complete  structure  has 
been  designed  to  withstand  normal  loads  without  failures  which 
will  result  in  loss. 


2 .  IDENTIFICATION  OF  CRITICAL  ITEMS 

As  we  mentioned  earlier,  the  identification  of  weaknesses  in  the 
design  is  not  the  end  objective.  From  the  analysis, we  must  determine 
corrective  action  to  improve  the  design.  The  failure  modes  and 
effects  analysis  can  be  used  to  assess  the  relative  importance 
of  the  various  weaknesses  isolated  to  permit  intelligent  appli¬ 
cation  of  effort  (time  and  money)  in  selecting  corrective  action. 
This  is  performed  as  follows: 

A  reliability  model  is  developed  for  the  system.  This  model 
serves  for  definition  of  the  subsystems  and  identification  of 
the  functional  components.  It  is  not  a  functional  schematic  or 
an  energy  flow  diagram,  but  serves  for  early  analysis  and  to 
point  out  "weak  links"  which  detract  from  the  overall  mission 
attainment.  A  model  of  a  system  should  have  provisions  to  point 
out  the  failure  modes  which  are  applicable  to  redundancy. 

2.1  FAILURE  EFFECT  ANALYSIS 


A  failure  mode  and  effect  analysis  is  performed  for  each  block 
in  the  reliability  analysis  logic  diagram.  The  failure  effect 
analysis  shall  indicate  the  effect  of  component  failure  on  the 
subsystem  or  system  performance.  In  determining  the  effect  of  a 
component  failure  on  sub-system  performance,  four  modes  of  fail¬ 
ure  are  considered. 

1.  Premature  operation  of  a  component. 

2.  Failure  of  a  component  to  operate  at  prescribed  time. 

3.  Failure  of  a  component  to  cease  operation  at  a  prescribed 
time. 

4.  Failure  of  a  component  during  operation. 


Each  component  is  evaluated  in  this  manner  for  the  failure  modes 
that  are  applicable. 
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A  usual  form,  Figure  12-12,  for  the  single  failure  effect  analysis, 
calls  for  the  following  critique  of  each  component  in  the  system: 


Column 

Nomenclature 

Description 

1 

Item 

Identify  item  by  name,  number  required 
and  code  designation. 

2 

Part  No. 

Federal  Stock  Number,  Classification  or 
Circuit  designation,  etc. 

3 

Function 

Concise  statement  of  the  components 
function. 

4 

Failure  Mode 

Concise  statement  of  the  applicable 
mode(s)  of  component  failure. 

5 

Failure  Effect 
on  System  Per¬ 
formance 

Full  explanation  of  the  effects  on  the 
performance  of  a  system  and  the  de¬ 
pendency  on  time  for  a  given  part 
failure  and  a  justification  of  the 
probability  of  loss  statement. 

6  Loss  Probability  Assign  numerical  index  for  the  profcab- 

{%)  ility  of  system  loss  if  part  fails. 

Suggested  scale  Certain  Loss  -  100%, 
Probable  Loss  -  50%,  No  Effect  -  0%. 

7  Failure  Mode  Enter  estimated  or  recorded  ratio  of 
Frequency  Patio  failures  in  each  mode  to  total  failures 

of  the  part. 

2.2  RELIABILITY  MODEL  INDEXING  NUMBERS 


A  means  for  direct  reference  of  all  items  in  the  reliability 
model  is  provided  by  using  an  indexing  number  system.  Numbers 
are  used  to  denote  systems,  subsystems,  assemblies  and  components. 
If  new  items  are  added  or  existing  items  removed,  new  numbers 
are  assigned  to  the  additions  and  the  existing  numbers  are  dis¬ 
continued  for  deleted  items. 

2.3  CRITICALITY  RANKING 

A  critical  items  list  is  by  definition  based  on  the  item's 
applicable  failure  mode,  tine  system  loss  probability  from  the 
failure  effect  analysis,  the  item's  failure  mode  frequency  ratio, 
and  the  item's  unreliability  associated  with  the  critical  failure 
mode  (or  modes). 
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The  failure  mode  frequency  ratio  is  determined  by  the  failure 
history  of  the  component.  The  failure  mode  frequency  ratio 
(FMFR)  is  the  ratio  of  the  number  of  failures  that  occur  in  a 
single  mode  to  the  total  number  of  failures: 

Failures  in  a  single  mode 
,UI ''  Total  number  of  failures 

If  a  failure  history  is  not  available  on  the  particular  component 
in  question  to  determine  the  failure  mode  frequency  ratio,  similar 
components  used  in  the  industry  are  valuable  sources  of  failure 
information.  Care  should  be  taken  that  the  similar  item  is  used 
in  a  similar  situation. 

The  unreliability  of  a  component  is  determined  from  its  failure 
rate  and  its  time  of  operation.  Appeal  is  made  to  the  System 
Model  to  determine  the  environmental  conditions  during  component 
operation  and  the  time  of  component  operation  per  mission  phase/ 
subphase.  The  time  of  operation  and  the  environmental  conditions 
must  be  known  to  predict  the  failure  rates  and  number  of  fail¬ 
ures  of  components.  Once  the  failure  rate  is  determined  it  is 
multiplied  by  the  length  of  time  of  operation  in  the  following 
equation  to  determine  the  unreliability: 


41  * 
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where 

X  =  failure  rate 
t  =  time  of  operation 

Criticality  Ranking  is  accomplished  by  multiplying  the  thi\,a 
factors  together : 

CR  =  (PL)  (FMFR)  (Q) 

where 

PL  =  probability  of  loss 

FKFR  =  Failure  mode  frequency  ratio 

Q  =  probability  of  component  failure 

2.4  CRITICAL  ITEMS  LIST 


Based  on  the  single  failure  effect  analysis,  a  critical  items 

A' 


list  is  prepared.  These  listings  are  an  abstract  of  those  items 
in  the  failure  effect  analysis  whose  single  failure  results  in 
the  probability  of  loss,  placed  in  numerical  sequence  of  their 
criticality  ranking.  The  form  for  these  critical  item  lists 
should  include  the  following  information: 


Column 

Nomenclature 

Description 

1 

Sys  tern 

Identify  system  by  indexing  number 

2 

Subsystem 

Identify  subsystem  by  indexing  number 

3 

Assembly 

Identify  Reliability  Functional  Block 
by  indexing  number. 

4 

Component 

Identify  component  by  indexing  number . 

5 

Item 

Identify  item  by  name 

6 

Mode  of  Failure 

Concise  statement  of  the  applicable 
mode  of  component  failure. 

7 

Loss  Effect 

The  degree  of  loss  probability  (should 
the  indicated  bype  of  failure  occur). 

8 

Reaction  Time 

The  estimated  time  elapsed  from  a  com¬ 
ponent  failure  to  loss  of  vehicle  (i.e 

0.1,  1,  10,  100,  etc.  seconds). 

9  Criticality  As  computed  in  paragraph  2.3. 

Ranking 

Where  a  component  has  more  than  one  mode  of  failure,  which  re¬ 
sults  in  the  probability  of  loss,  separate  entries  are  made  in 
the  critical  items  list  for  each  mode. 

Criticality  ranking  or  classification  has  the  same  basic  context 
as  the  "Levels  of  Essentiality"  criteria  for  design,  materials 
control  and  traceability  in  submarine  pressure  boundaries  (Refer¬ 
ence  8)  . 

2.5  APPLICATIONS  OF  CRITICALITY  RANKING 


The  numerical  value  of  the  criticality  ranking  orders  the  com¬ 
ponents  by  the  degree  to  which  they  are  expected  to  create  pro¬ 
blems.  A  high  ranking  number  indicates  that  the  particular  mode 
of  the  component  needs  special  attention  in  the  design  and,  if 


it  cannot  be  reduced,  particular  attention  paid  to  the  component 
in  manufacture  and  use.  In  this  way  the  failure  mode  analysis 
may  be  used  to  sort  out  the  problems  involved  in  a  development 
program  to  focus  attention  on  those  of  the  greatest  importance. 

There  isn't  one  of  use  who  has  not  been  faced  with  the  problem 
of  a  program  with  too  little  money  and  too  little  time  to  do  the 
job.  The  analytical  technique  presented  gives  early,  realistic 
discrimination  criteria  which  provide  greatest  assurance  the  pro¬ 
gram  will  meet  it^  requirements  with  the  most  effective  money 
expenditure.  Given  critical  items,  failure  modes,  and  criticality 
ranking  the  components  designer  becomes  concerned  with  their 
application  in  many  areas. 

Criticality  ranking  should  be  used  to  establish  which  items  should 
be  first  to  receive  review.  It  should  be  used  to  establish  the 
specific  areas  of  invest iaation  in  a  design  review.  The  review 
should  emphas-Z’'  where  possible  "fail  safe"  operation  for  cri¬ 
tical  mode,.  Where  Ihis  is  not  possible  redundancy,  override 
controls,  and/or  failure  sensing  devices  should  be  incorporated. 
Since  no  proaram  i.‘  infinitely  funded  it  becomes  apparent  that 
the  "totem  f  le"  established  by  criticality  ranking  provides 
technical  and  management  judgment  criteria  for  where  best  to 
spend  program  money. 

The  designer  m  y  use  critical  items  to  establish  which  supplier 
specifications  should  have  more  stringent  than  normal  require¬ 
ments  for  design,  monitoring,  and  test  imposed.  Since  effectively, 
the  components  will  undergo  very  little  change  once  the  supplier 
has  delivered  an  approved  part  to  the  system,  it  is  imperative 
that  the  design  reflect  minimum  critical  failure  mode  probability. 
Additionally,  the  supplier  test  program  should  reflect  stringent 
consideration  of  these  characteristics.  Such  te3t  programs  should 
analyze  the  effects  cf  combined  environmentax  and  critical  opera¬ 
tional  stresses  on  the  hardware  in  order  that  the  interaction  of 
environments  on  the  hardware  will  be  properly  investigated. 

Criticality  ranking  is  an  excellent  discrimination  criterion  in 
that  it  will  give  the  best  return  for  traceab^ity  per  dollar 
invested.  If  program  money  is  too  short  to  provide  traceability 
on  all  critical  items,  the  criticality  rankinq  index  should  be 
u  ed.  For  instance  only  those  items  with  a  criticality  ranking 
in  the  upper  1S%  mighl  be  made  traceable. 

Screening  specifications  can  be  established  by  the  designer  to 
assure  that  any  components  classed  as  critical  entering  the  plant 
will  be  given  a  prescribed  test  or  inspection  for  particular 
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weakness.  Tho  items  to  be  screened  should  be  selected  from  the 
total  critical  items  lis  ,  i f  program  money  is  limited,  more 
discriminating  selection  c  n  be  made  from  the  criticality  rank¬ 
ing.  The  characteristics  to  be  inspected  should  be  taken  from 
the  failure  effect  analysis. 

Finally,  the  component  designer  should  establish  that  the  failure 
reporting  system  which  exists  in  his  company,  specifically  re¬ 
ports  failures  on  all  critical  items  as  such.  He  should  also  see 
that  the  reporting  system  stipulates  the  specific  mode  in  which 
the  component  failed.  The  critical  items  list  should  be  used  to 
establish  which  items  will  receive  special  expedited  attention 
in  the  failure  reporting  and  corrective  action,  system.  Provision 
should  be  incorporated  into  the  reporting  system  for  directly 
identifying  on  the  report  those  failure  modes  which  have  been 
established  by  failure  effect  analysis  as  c- itical.  With  this 
type  of  information  plus  the  normal  reliab  lity  statistical  in¬ 
formation  surrounding  failures  and  failure  ans lysis,  we  can  go 
back  to  the  reliability  model  and  spec  fi  illy  report  in  critical 
areas  what  has  in  fact  happened.  This  p.  vi  ies  for  a  much  more 
expedited  and  meaningful  analysis. 


5.  REDUCING  EFFECTS  OF  FAILURE 

Failure  Modes  and  Effects  Analysis  is  a  design  analysis  tool  used 
by  design  and  reliability  engineers  to  measure  the  probabilities 
of  losses  associated  with  failures  in  a  system  design. 

After  the  failure  mode  and  effect  analysis  has  been  completed, 
specific  items  should  be  summarised  to  indicate  where  redesign 
would  improve  the  reliability  through  consideration  of  physical 
phenomena  associated  with  the  potential  failure.  The  redesign 
may  include  additional  margins  of  safety,  change  of  materials, 
process  controls,  environmental  control,  or  specialized  testing 
to  inhibit  or  control  that  particular  mode  of  failure. 

In  order  to  provide  a  basis  for  loss  reduction  task  ,  the  de¬ 
signer  systematically  ranks  the  failures  in  teim^  of  heir  pro¬ 
babilities  of  failure  and  their  associated  losses.  Action  is 
taken  to  prevent  the  occurrence  of  high  loss  failures.  High  loss 
failures  are  attacked  by  the  following  schemes: 

1.  Schemes  to  prevent  component  failures.  These  s-b ernes  in¬ 
volve  : 

a.  Redesign  which  accomplishes  one  or  more  of  the  following: 


1)  Reduce  the  cause  of  failure 

2)  Design  around  failure  mode 

3)  Reduce  the  effect  of  failure 
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b.  Modify  maintenance  schedules  or  instructions 

2.  Schemes  to  prevent  the  propagation  of  failure  effects - 
These  schemes  involve: 

a.  Monitoring  to  detect  component  failures  whose  effects 
may  cause  a  loss  event,  and  give  suitable  warning. 

b.  Counteraction  which  accomplishes  one  or  more  of  the 
following : 

1)  Nullifies  the  effects  or  conditions  leading  to  loss 
events  or  protects  agains  them.  (This  includes 
crew  escape,  for  example) . 

2)  Controls  or  deactives  components,  systems,  etc.,  so 
as  to  halt  generation  or  propagation  of  harmful 

e  f fects  . 

')  Activates  backup  or  standby  units  or  systems  to 
restore  interrupted  functions. 

4)  Replaces  failed  components  if  practicable 

4.  SUMMARY 

The  a1  erally  accepted  definition  of  reliability  implies  the 
assignment  of  a  function  or  set  of  ta.‘ ks  for  tl  e  equipment  and 
associi  ed  personnel  to  perform.  Also  implied  is  the  definition 
of  a  failure  state  or  mode  for  each  task,  so  that  the  probability 
of  a  system  being  in  one  or  the  other  of  two  exclusive  states, 
success  ,  -  failure  after  some  period  of  time,  may  be  estimated. 
Mechanical  reliability  is  much  more  a  conditional  probability 
tnan  we  are  used  t  o  considerino  it  for  electronics .  The  condition 
applied  is  the  probability  of  system  failure  given  chat  component 
failure  occurs.  Failure  modes  and  effects  analysis  is  the  meth¬ 
odical  evaluation  of  this  condition.  The  future  path  of  relia¬ 
bility  analysis  will  include  studies  in  depth  on  the  physics  of 
failure  (91  using  techniques  such  as  the  Failure  Modes  &  Effects 
Analyses  to  improve  our  capability  of  reliable  uesign. 
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DESIGN  FOR  RELIABILITY  AND  MAINTAINABILITY 

The  requirement  for  very  high  reliability  in  weapons  systems  and 
critical  systems  aboard  ship  runs  into  two  roadblocks  —  cost  of 
achievement  and  time  required  for  development.  In  establishing 
the  minimum  acceptable  reliability,  the  only  realistic  initial 
basis  is  the  amount  of  risk  (that  the  system  will  not  work  when 
required)  that  the  operational  commander  is  willing  to  accept. 

Once  the  risk  is  established,  then  the  decision  to  proceed  with 
the  development  —  or  cancel  it  —  will  be  made  on  the  basis  of 
priority  of  need,  cost  and  time.  If  the  acceptable  risk  is  too 
costly  or  takes  too  long  to  get,  it  may  be  desirable  to  trade  re¬ 
liability  for  other  performance  capability. 

As  we  have  pointed  out  before,  the  efficient  way  of  achieving 
required  reliability  is  in  the  initial  design.  If  this  can  be 
done,  it  eliminates  many  of  the  costs  and  delays  associated  with 
improving  the  design  to  meet  the  requirements  after  production 
starts  and  many  of  the  costs  of  problems  associated  with  unrelia¬ 
bility  of  the  equipment  in  operational  use  (ownership  costs). 

Reliability  can  be  improved  by  the  designer  before  the  equipment 
is  constructed,  before  the  design  is  released.  This  chapter  will 
discuss  the  accepted  approaches  the  designer  can  use  to  achieve 
the  reliability  requirement  once  he  has  ascertained  that  it  will 
not  otherwise  be  met  in  the  design.  Here  are  the  steps  involved* 

1.  Verify  Stated  Requirements*  Seldom  does  the  bald  quantitative 
statement  of  required  reliability  and/or  maintainability  actually 
convey  the  picture  needed  by  the  design  engineer.  There  must  be 
thorough  discussion  of  what,  physically  and  specifically,  is 
meant  by  the  numbers.  Such  discussion  usually  results  in  further 
definition,  if  not  actual  change  of  the  stated  requirement. 

I  ,  ./■ 

2.  Define  Unacceptability  *  A  reliability  requirement  has  no 
meaning  until  a  very  clean  answer  is  .obtained  to  the  question 
"What,  exactly,  constitutes  a  failure?"  This  is  particularly 
difficult,  and  important,  regarding  the  slow  degradation  of  per¬ 
formance  found  in  most  systems.  How  much  can  a  certain  perform¬ 
ance  value,  say  accuracy,  degrade  before  it  is  judged  a  failure? 
Often  it  is  far  more  logical  to  use  broader  effectiveness  criteria, 
such  as  "fire-power"  (1)  instead  of  the  black  snd  white  success  or 
failure.  The  same  comments  can  be  made  about  maintainability, 
i.e.,  "What,  exactly,  constitutes  excessive  downtime?"  And  "Why?" 
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3.  Design  for  Required  Reliability:  When  the  requirement  (and 
meaning  of  failux'e)  is  understood,  the  designer  proceeds  with 
tentative  design,  maintaining  a  current  record  of  the  status  of 
predicted  reliability  achievement  but  weighing  each  decision  he 
must  make  against  the  effect  that  the  alternative  selected  will 
have  on  the  reliability,  in  the  same  way  that  he  does  weight,  per¬ 
formance  capability,  cost  and  delivery.  His  objective  must  be 

to  find  the  optimum  combination  of  performance  capability,  weight, 
cost,  delivery  and  reliability  that  meets  his  functional  require¬ 
ment.  Excess  capability  over  that  minimum  requirement  must  be 
weighed  against  consequences  in  terms  of  the  constraints  of  cost, 
delivery,  etc.  If  the  tentative  design  for  functional  capability 
fails  to  achieve  the  required  reliability.  Th€  ”>ecial  techniques 
covered  in  this  chapter  should  be  applied  to  the  cinalysis. 

4.  Cost-Effectiveness  Analysis :  Once  the  design  is  complete  to 
the  extent  of  specifying  major  components  in  detail,  a  rough  cost- 
effectiveness  analysis  should  be  made,  using  the  techniques  of 
chapter  26,  and  estimating  the  cost  and  time  to  design  and  manu¬ 
facture  for  the  tentative  reliability  and  maintainability  require¬ 
ment  values  of  MTBF  and  MTTR  are  selected  to  optimize  the  cost 
effectiveness  relationship., 

i 

5.  Modification  and  Recycle t  As  the  design  progresses  to  more 
detailed  component  level,  decisions  will  be  made  affecting  the 
reliability  and  maintainability.  These  must  be  evaluated  against 
their  effects  on  total  cost ana  performance  t  cC^uir  cmcuuS  |  mOdl  fying 
the  requirements  as  necessary  to  optimize  the  cost-effectiveness 
achieved. 


1.  BASIC  RELIABLE  DESIGN 

As  a  result  of  many  years  and  cycles  of  product  design,  manufac¬ 
ture,  operational  experience,  and -consequent  design  improvement, 
most  contractors  have  built  up  a  comprehensive  set  of  standard 
practices.  These  practices  assure  "good"  design  by  the  traditional 
criteria. 

'••Ait  the  criteria  have  changed.  Military  product  complexity  has 
made  the  previous  standards  of  reliability  unacceptable.  The 
loss  of  system  effectiveness,  the  excessive  maintenance  cost, 
and  the  unavailability  pfj ‘^maintenance  skills  have  demanded  new 
reliability  criteria.  :;  ** 

So  in  addition  to  the  established  standard  "good"  engineering 
practices,  which  cure  beyond  our  scope  here,  basic  reliable  design. 


13-5 


demands  the  additional  formalized  practices,  many  of  which  are 
extensions  of  standard  "good"  engineering  practices,  outlined  in 
this  section.  These  are  the  techniques  to  he  applied  to  every 
design  to  a  reliability  requirement. 

1.1  SIMPLIFICATION  TECHNIQUES 

Hardly  anybody  doubts  that  the  way  to  get.  real  reliability  is  to 
make  it  simple.  Like  an  ash  tray.  Yet  often  it  will  not  occur  to 
some  computer  people  that  a  slide  rule,  or  a  pencil  and  paper, 
may  be  adequate  and  more  reliable  for  some  tasks  than  e.  computer. 

Or  to  system  designers  that  a  hydraulic  or  mechanical  system  may 
be  much  simpler  and  more  relie.ble  for  some  tasks  than  an  electrical 
system.  Or  vice  versa. 

\  ' 

The  average  design  engineer  can  get  preoccupied  with  elegance. 

He  has  been  so  encouraged  to  dream  of  new  ways  to  get  more  per¬ 
formance  capability,  without  much  regard  to  reliability  or  cost, 
that  complexity  is  accepted  as  inevitable.  But  it  isn't. 

For  many  years  now  the  "value  analysis"  techniques  have  enjoyed 
growing  recognition  and  acceptance,  and  they  have  produced  re¬ 
markable  cost  reductions.  Generally  the  procedure  is  to  put  a 
team  to  work  on  a  released  (in  manufacture)  design,  with  leader¬ 
ship  and  instructions  (2)  through  the  "information,  creative, 
evaluation,  investigation,  and  reporting"  phases.  True  value 
analysis,  where  adequate  performance,  including  Reliability,  5. s 
maintained  by  a  simpler  or  less  expensive  equipment,  is  not  the 
same  as  "cost  reduction"  which  may  accept  reduced  performance  or 
even  reduced  reliability  to  achieve  cost  savings.  / 

Virtually  all  such  analyses  have  been  applied  to  existing  designs, 
to  reduce  manufacturing  cost  through  substantial  simplification. 

But  many  times  a  substantial  potential  cut  in  manufacturing  cost 
would  be  offset  by  an  increased  logistic  and  maintenance  cost,  iso 
the  customer  cannot  approve  it.  The  real  objective  to  the  user 
is  not  manufacturing  cost  reduction,  but  total  cost-effectiveness 
improvement.  And  reliability  is  a  major  effectiveness  element. 

Many,  if  not  most,  such  analyses  result  in  reliability  improve¬ 
ment,  usually  as  a  byproduct  of  simplification.  So  it  becomes 
obvious  that  the  same  techniques  can  be  used  for  deliberate  re¬ 
liability  improvement.  Following  is  a  typical  "value-engineering" 
phase  description,  but  modified  to  achieve  optimal  reliability* 

Information  Phase:  Obtain  full  information  on  the  design  require¬ 
ments,  distinguishing  the  mandatory  from  merely  desirable,  and 
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analyse  relative  to  realistic  needs  and  constraints.  Specifically 
include  reliability  and  maintainability,  obtain  full  information 
on  the  proposed  or  released  design,  including  predicted  reliability 
and  maintainability,  and  acquisition  (design  &  manufacture)  and 
ownership  (operation  and  maintenance)  coots,  using  the  best  avail¬ 
able  sources. 

Determine  the  basic  and  secondary  functions  of  the  design,  using 
verb-noun  definitions  ("transmit  torque",  "protect  surface", 

"conduct  current",  etc.)  Segregate  portions  of  functions  into 
sequence. 

Creative  Phase:  Use  the  "brainstorming"  technique  (3)  to  list 
all  possible  alternative  ways  of  performing  the  required  functions 
defined  above.  Avoid  negative  ("It  won't  work")  judgment  while 
generating  as  many  simple  and  direct  ideas  as  possible,  and  re¬ 
cording  them.  Group  action  is  necessary  for  triggering  ideas  in 
each  other. 

Evaluation  Phases  Evaluate  each  of  the  above  ideas  on  the  basis 
of  effectiveness,  or  the  best  reliability  and/or  availability 
that  satisfies  functional  per formance ’requirements .  Then  evaluate 
each  for  total  cost  of  design,  manufacture,  operation,  logistics, 
and  maintenance  over  the  system  useful  lifetime.  Consult  with 
specialists,  suppliers,  and  the  customer  as  necessary.  Don't  re¬ 
invent  something  already  available,  if  the  available  one  is  adequate. 

Finally  reconstruct  the  list  in  decreasing  order  of  apparent  ratio 
of  effectiveness  to  total  cost,  using  quantitative  evaluations 
where  feasible.  Such  ordering  decisions  are  usually  meaningful 
only  when  comparisons  can  be  made. 

Investigation  Phase;  Using  the  above  basic  and  secondary  functions, 
determine  the  reliability,  maintainability,  and  total  cost  of  each. 
Compare  these  with  target  values  obtained  from  other  applications 
of  the  same  function.  Consider  all  standard  components  available. 
Work  on  specifics,  not  generalities.  Select  the  best  one  or  two 
ideas  on  the  basis  of  the  detailed  analysis. 

Reporting  Phase:  Provide  a  concise  report  for  design  use  and 
documentation,  including  all  data  sources,  analyses,  and  logic 
leading  to  the  selection. 

Reference  (2)  provides  a  checklist  to  indicate  some  approaches 
for  value  engineering  ideas. 

Above  <ill,  there  is  far  greater  advantage  in  application  of  these 
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techniques  long  before  release  to  manufacture,  after  which  change 
becomes  an  order  of  magnitude  or  two  more  costly.  They  can  be 
used  by  conceptual,  system,  and  component  design  engineers  on 
purely  paper  designs. 

1.2  STANDARDIZATION 

There  is  a  place  in  research  and  development  for  new  ideas,  but 
once  the  state  of  the  art  is  advanced,  the  development  should  be 
based  on  system  effectiveness.  Many  design  engineers  have  resisted 
standardization,  on  the  ground  that  it  restricts  their  freedom 
for  exercise  of  unbridled  creativity  and  "progress"  to  new  things. 
Now  that  we  have  altered  the  objective  from  "new  things"  to  "things 
that  keep  working",  such  resistance  amounts  to  poor  engineering. 

•i 

But  unless  such  standards  are  kept  vigilantly  up  to  date  with 
advancing  state  of  the  art,  they  can  discourage  initiative  for 
new  developments.  They  must  be  constantly  reviewed  to  add  new  ’ 
standards.  In  the  case  of  new  physical  hardware  standards ,  very 
thorough  reliability  verification  must  precede  their  establish¬ 
ment  prior  to  withdrawal  (for  new  design)  of  obsolete  standards. 

The  American  Standards  Association  (4)  has  been  established  for 
national  approval  of  standards  sources  and  for  distribution  of 
many  standards.  Now  let's  review  those  pertinent  to  our  needs. 

1.2.1  Standard  Values :  There  is  often  quite  substantial  econ¬ 
omic  and  reliability  benefit  in  the  establishment  of  standard 
sizes  and  values  to  be  used  by  all  contractors  and  the  government 
to  mutual  advantage.  Chapter  18  provides  some  detailed  examples 
for  parts.  Another  very  familiar  example  is  screw  sizes. 

Bat  the  same  principle  can  apply  at  any  level.  We  have  largely 
standardized  desk  heights  at  29" .  Automobile  Widths  are  fairly 
standard.  Electrical  power  systems  operate  at  quite  standard  , 
voltages  and  frequencies.  The  result  is  higher  quantities ,  and 
better  testing  of  any  one  standard  design,  thus  better  reliability. 

1.2.2  Standard  Parts  t  The  establishment  of  standard  parts 
designs  can  provide  a  manufacturers  cost  reduction,  higher  relia¬ 
bility,  contractors  cost  reduction,  ownership  cost  reduction, 
better  operational  data,  and  better  control  of  tolerance  limits. 
Details  are  given  in  chapter  18. 

1.2.3  Standard  Components  ?  (such  as  regulator  valves  or  ampli¬ 
fier  circuits)  can  be  either  selected  from  available  supplier 
products  or  developed  by  the  design  engineer,  for  wide  use  across 
a  range  of  higher-level  designs.  MIL  STD  242E  provides  standard 


components  for  electronics  use. 

The  above  considerations  for  parts  apply  equally  well  to  components, 
though  there  are  fewer  such  standards  and  they  are  more  complex. 
Many  companies  maintain  a  file  of  thoroughly- proven  circuits, 
which  may  be  either  used  directly  or  modified  to  avoid  the  unre¬ 
liability  and  cost  of  complete  reinvention.  But,  sadly,  the  amount 
of  such  reinvention  in  the  U.  S.  must  be  staggering,  for  sheer  lack 
of  communication  and  other  reasons. 

1.2.4  Standard  Systems,  Subsystems  and  Major  Components  (such 
as  a  hydraulic  servo  system),  made  up  of  components,  can  likewise 
be  established  and  used  across  many  higher-level  systems.  The 
Air  Force  has  established  a  "standard  launch  vehicle"  for  this 
reason.  Reliability  improvement  always  results. 

1.2.5  Standard  Design  Methods  (such  as  hull  girder  strength)  can 
be  established  for  mandatory  use  by  design  engineers.  Over  a 
period  of  more  than  100  years,  by  reiterative  sequential  correc¬ 
tion  and  improvement  of  design  as  errors  and  problems  are  identi¬ 
fied  in  actual  operational  use,  many*technologies  have  developed 
standard  "rules",  "codes",  specifications,  etc.  that  are  very 
widely  accepted.  Basically  they  are  empirical  rules  that  result 
in  high  quality,  reliability,  and  safety. 

Competition  eventually  prevents  them  from  approaching  overdesign, 
except  to  the  extent  that  they  sometimes  lag  state-of-the-art 
material  technology.  Nevertheless ,  adherence  to  such  rules  and 
codes  does  assure  "high"  reliability  and  safety,  but  does  not 
necessarily  achieve  the  best  or  optimum  value  of  reliability  in 
relation  to  acquis i-tion  and  ownership  cost.  Here  are  three 
examples . 

Rules  for  Building  and  Classifying  Steel  Vessel*  (5)  is  tn  excel¬ 
lent  compendium  of  rules,  containing  the  following  subjects* 

a.  Rules  for  Construction  &  Classification  of  Steel  Vessels 

b.  Rules  for  Construction  &  Classification  of  Machinery 

c.  Rules  for  Inspection  and  Testing  of  Materials 

d.  Rules  for  Fire  Pumps  and  Fire  Extinguishing  Systems 

e.  Rules  for  Surveys  after  Construction 

f.  Tables  of  Scantlings 

g.  Tables  of  Equipment 

h.  Load  Line  Markings 

ASME  Boiler  and  Pressure  Vessel  Code  (6)  has  the  objective  of 
providing  "reasonably  certain  protection  of  life  &  property,  and 


13-9 


to  provide  a  margin  of  deterioration  "  (wear out  reliability)''  in 
service  so  as  to  give  a  reasonably  long  "(reliability)"  safe  period 
of  usefulness.  Advancements  have  been  recognized.  Interpretations 
are  published  in  the  magazine  Mechanical  Engineering  as  "Code 
Cases".  The  major  sections  are 

1.  Power  Boilers 

2.  Material  Specifications 

4.  Low-Pressure  Heating  Boilers 

7.  Suggested  Rules  for  Care  of  Power  Boilers 

8.  Unfired  Pressure  Vessels 

9.  Welding  Qualifications 

National  Electrical  Safety  Code  (7 )  applies  to  ground  installations 
rather  than  shipboard,  and  is  legally  binding  in  most  uA  S.  muni¬ 
cipalities.  It  is  approved  by  the  American  Standards  Association  .... 
(4)  as  an  American  Standard.  Decisions  are  made  by  sectional  com¬ 
mittees,  and  approved  by  the  American  Standards  Association.  Its./ 
content  is  s  -  \  ' 

1.  Rules  for  the  installation  and  maintenance  cf  electrical 
supply  stations. 

2.  Rules  for  the  installation  ahd  maintenance  of  electric 
supply  and  communication  lines . 

3.  Rules  for  the  installation  and  maintenance  of  electric 
utilization  equipment  (conductors,  fuses,  circuit  breakers, 
motors  &  machinery,  storage  batteries,  transformers, 
lighting,  appliances,  cranes,  elevators,  telephone  appar¬ 
atus  ) . 

4.  Rules  for  the  operation  of  electric  equipment  and  lines. 

5.  Rules  for  radio  installation. 

1.2.6  Standard  Analysis  Methods  (such  as  reliability  prediction) 
can  be  established  for  applicable  use  by  design  and  reliability 
engineers.  Such  methods  are  covered  in  other  chapters  of  this 
course.  Some  government  agencies  and  contractors  h?ive  attempted 
to  establish  specifications  and  mandatory  analysis  techniques 
that  work  nicely  for  some  limited  scope  of  problem,  but  which  do 
not  work  for  many  other  problems.  Analysis  standardization'  is 
useful  to  the  extent  that  its  applicability  limitations  arc! 
recognized. 

1.2.7  Drafting  Standards  help  to  assure  that  drawings  and  v  L 

specifications  are  consistent,  legible,  and  complete,  thus  mini¬ 
mizing  human  error  and  consequent  system  unreliability. 
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1,2.6  The  Military  Standards  System  (Mil-Std)  provides  many 
standards  tor  computation,  analvsis  and  management.  For  the 
manor  reliability  and  maintainability  standards,  see  Chapter  17. 

1.3  STRESS/ STRENGTH  DESIGN 

The  classical  and  completely  valid  approach  to  desian  is  to  give 
every  part  enough  strength  to  handle  the  worst  stress  it  will 
encounter.  Hundreds  of  books  such  as  Mil-Hbk-5  (10)  are  avail¬ 
able  providing  data  on  the  strength  of  materials,  and  some  of 
these  provide  limited  data  on  strength  degradation  with  time, 
resulting  from  fatigue. 

But  when  we  come  to  design  for  a  specified  reliability,  the 
traditional  a. id  common  use  of  "safety  factors"  and  "safety  margins'* 
is  inadequate.  We  have  to  design  in  such  a  way  that  we  can  at 
least  roughly  predict  either  (a'  the  MTBF  of  the  design  in 
operational  use,  or  failing  that  (b)  the  probability  that  stress 
will  not  exceed  strength.  At  least  three  approaches  have  been 
developed : 

1.3.1  Derating :  Intuitively  every  design  engineer  feels  that 
reliability  is  improved  by  using  parts  rated  much  higher  than  the 
expected  stress.  That  is,  he  "derates"  the  parts  tor  his  appli¬ 
cation.  Tt  is  equivalent  to  increasing  the  "safetv  factor". 
Unfortunately  this  practic  also  increases  cost,  weight,  and 
volume.  If  operational  experience  shows  no  failures,  he  never 
knows  how  much,  if  any,  unnecessary  cost,  weight  and  volume  he 
has  incurred.  We  are  all  aware  of  such  examples  of  "overdesign". 
Nevertheless,  judicious  derating  is  a  powerful  aid  to  reliability. 
Parts  derating  is  covered  in  Chapter  18. 

1.3.2  Reliability  Margin:  In  the  absence  of  adequate  failure 
rate  data,  which  absence  is  common  in  mechanical  and  structural 
fields  to  date,  a  second  approach  is  available.  Robert  Lesser 
originally  ,-rcpctcd  it  in  ICC7.  Keeecioglu  recently  published 

a  technical  summary  (1C)  of  the  techniques,  with  many  references. 
The  method  is  covered  in  detail  in  Chapter  7. 

Nearly  all  design  involves  many  assumptions  to  avoid  uniusti  li¬ 
able  volume  of  analysis  or  test  cost.  The  outlined  approach 
permits  design  to  predictable  reliability,  b”t  does  not  insure 
against  design  errors  of  assumption,  analysis,  omission,  etc. 
Verification  is  mandatory  if  high  reliability  is  to  be  achiev  d. 
Possible  activities  in  the  verification  area  should  include: 

1.  Conduct  independent  Reliability  Margin  analysis:  An 
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independent  reliability  analysis  of  the  design,  by 
analysts  other  than  those  who  conducted  the  design  pre- 
diction,  pays  dxv’jends.  A  fresh  viewpoint,  alternative 
analytical  methods,  etc.,  nearly  always  turn  up  details 
worth  changing  before  the  design  is  too  far  committed  to 
manufacture. 

2 .  Conduct  tests  to  failure  for  critical  margins?  When  the 
above  design  approach  is  used,  probably  all  critical 
margins  have  become  well-known  to  the  design  engineer. 
Listing  the  first  dozen  or  two  of  these  in  the  order  of 
increasing  Reliability  Margin,  he  can  then  estimate  what 
a  series  of  simple  tests  to  failure  for  each  would  in¬ 
volve  in  cost  and  time.  Or  it  may  be  feasible  and  more 
conclusive  to  design  fewer  tests  to  failure  of  a  higher 
system  level  to  achieve  the  required  verification.  Ten 
lunar  Excursion  Module  prototypes,  for  example,  will  be 
used  for  such  tests  to  failure. 

3.  Modify  the  design  and/or  material:  Independent  analysis 
and  test  to  failure*  are  of  course  worthless  until  their 
lessons  are  translated  to  design  improvement.  Surprisingly, 
this  is  sometimes  resisted.  „ 

1.3.3  stress/Strength  Testing:  When  distribution  data  is  not 
obtainable  for  the  above  analytical  approach,  yet  the  design 
reliability  in  a  critical  matter,  it  may  be  necessary  to  conduct 
experimental  tests.  Tests  to  determine  stress  distribution  in  a 
prototype  are  fairly  straightforward  and  non-destructive,  using 
instrumentation  such  as  strain  gages,  plastic  models  ar.d  polarized 
light,  etc.  To  the  extent  that  such  tests  can  simulate  the 
manufacturing  variances,  operational  environment,  external 
stresses,  and  time  effects,  the  results  can  be  quite  dependable. 

But  tests  of  strength  distribution  are  much  more  difficult, 
expensive,  and  time  consuming.  If  the  design  engineer  can  identify 
specific  local  areas  of  critical  doubt,  a  series  of  comparatively 
simple  tests  can  be  designed,  wherein  stress  is  repeatedly  in¬ 
creased  until  failure  occurs,  providing  a  rough  strength  distri¬ 
bution  curve  for  the  local  area.  On  the  other  hand  it  may  be  more 
convincing,  if  not  more  economical  to  test  an  entire  prototype  in 
the  same  manner,  so  that  all  interactions  are  accounted  for, 
repairing  failures  each  time  they  occur.  Of  course  as  strength 
inadequacies  are  thus  brought  to  light,  the  design  is  changed  to 
get  required  strength. 

Such  stress/strength  testing  should  not  be  confused  with  simple 
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4  overs  tress '  testing,  which  determines  only  that  the  design  does 
not  fail  at  some  specified  stress  above  the  operational  level. 
Overstress  tesoir.g  does  not  generally  determine  strength. 

1.4  TOLERANCE  EVALUATION 

In  quanti'y  manu factor e ,  all  parts  characteristics  have  statistical 
distributions.  That  is,  any  one  characteris tic  (such  as  length  or 
resistance)  has  a  nominal  or  mean  value,  and  a  variance  above  and 
below  it.  We  call  the  extreme  values  of  the  variance  'tolerances". 
These  distributions  are  basically  affected  by  manufacturing  lot, 
ana  by  techniques  for  selection  of  close- tolerance  parts  out  of 
wide-tolerance  lets. 

In  addition  to  such  manufacturing  variance  there  is  application 
variance  regardless  of  quantity.  That  is,  there  are  distributions 
of  each  characteristic  resulting  from  environment  (temperature,  etc.) 
stress  {pressure,  voltage,  etc.),  and  time  (cold  flow,  drift,  aging, 
etc.).  Such  distributions  or  tolerances  must  be  added  to  the  manu¬ 
facturing  distributions  or  tolerances  in  order  to  determine  the 
real  operational  distribution. 

A  design  is  never  complete  until  the  design  engineer  has  made  sure 
that  the  distributions  or  tolerances  cannot  combine  in  such  a  way 
as  to  interfere  with  the  intended  function.  In  a  complex  circuit, 
mechanism,  or  structure  it  is  necessary  to  co. ider  the  overall 
effect  of  the  expected  range  of  manufacturing  variance,  operational 
environment  and  all  stresses,  and  the  effect  of  time.  Three 
general  types  of  evaluation  are  used  for  this  ,-urpose : 

1.4.1  Worst-Case  Tolerance  Analysis;  For  maximum  producibiiity 
and  reliability  the  design  engineer  often  attempts  to  design,  the 
equipm  .it  to  perform  properly  witn  all  parts  simultaneously  at  their 
tolerance  limits,  and  in  such  a  direction  as  to  produce  the  greatest 
deviation  of  nominal  performance.  For  relatively  simple  configura¬ 
tions  this  is  usually  easy  to  do,  and  quit^.  effective. 

But  for  the  more  complex  mechanisms  and  circuits,  such  an  attempt 
will  often  fail  because  even  the  best  and  highest  precision  parts 
will  not  have  small  enough  tolerances.  In  other  cases  the  tolerance 
problem  may  be  so  solved,  but  at  the  expense  of  complicating  the 
mechanism  or  circuit  to  the  extent  that  overall  reliability  suffers. 

Some  standard  computer  programs  are  available  for  such  worst-ca3e 
analysis  <^f  complex  systems  But  it  snould  be  kept  in  mind  that 
worst-cast  ina lysis  computes  a  situation  which  will  probably  never 
occur,  ana  which  therefore  leads  to  tighter  tolerances,  higher 
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rnanu  f  act  ur  inc:  coses  ,  and  usual  ly  dinner  reliability  than  are 
really  needed.  The  design  vi  11  be  extremely  reliable,  but  not 
very  cos t-e  f feet ive. 

1.4.2  Statistical  Tolerance  Analysi-.  Fortunately  the  prob¬ 
ability  that  all  the  parts  will  exist  at  their  maximum  tolerances 
simultaneously  is  very  remote.  Let  us  investigate  the  manner  in 
which  the  individual  parts  tolerances  afft  t  the  over-all  toler¬ 
ance.  This  effect  of  individual  tolerance  forms  the  basis  far  the 
statistical  approach  to  circuit  design.  The  following  material 
is  from  reference  (11)  which  is  an  abstract  of  detail  procedures 
in  reference  (8).  See  also  references  (12,  13,  14). 

It  is  well  known  that  many  production  parts  have  a  normal  or 
Gaussian  frequency  distribution  as  illustrated  in  the  following 
example.  Suppose  that  measurements  were  Made  of  the  values  of  a 
large  quantity  of  capacitors  of  the  same  nominal  value  and  +  10% 
tolerance.  Plotting  vertically,  the  number  of  capacitors  in  each 
1%  interval  of  capacitance  will  usu  lly  result  in  a  histogram 
similar  to  the  one  shown  in  Figure  Is  14-As  the  quantity  of  capaci¬ 
tors  measured  is  increased  and  the  capacitance  interval  is  narrowed, 
the  envelope  of  the  histogram  will  form  a  normal  distribution 
curve  as  shown.  This  curve  is  symmetrical  about  the  average  and 
asymptotic  at  the  base. 

The  total  area  under  the  curve  represents  all  the  capacitors.  The 
area  bounded  by  +p  covers  68 . 3%  of  the  total  area;  that  is,  66.3% 
of  the  capacitors  are  included  by  +o .  About  95.5%  of  the  capacitors 
are  included  by  +_2a  and  99.7%  are  included  by  +3 cf.  The  manufactur¬ 
ing  tolerance  will  usually  correspond  to  +  3c?  or  greater,  depending 
upon  the  degree  of  production  control;  that  is,  0.3%  or  less  of 
the  parts  usually  will  be  out  of  tolerance.  Of  course,  additional 
variations  in  capacitance  will  result  when  the  capacitors  are  sub¬ 
jected  to  conditions  of  operation  and  environment. 

Even  when  individual  parts  values  are  not  normal.!/  distributed, 
their  associated  circuit  output  variations  will  be  nearly  normally 
distributed  because  of  combination  effects- 

It  is  important  that  engineers  design  hardware  that  meets  tolerance 
specifications  in  a  very  high  percentage  of  the  equipments  built, 
both  from  production  and  reliability  standpoints.  To  illustrate 
with  an  oversimplified  example,  cc  -  der  a  circuit  >h i ch ,  in  pro¬ 
duction,  must  meet  a  tolerance  of  97  to  103  volts  output  (Figure 
13-15?.  If  the  actual  design  allows  production  to  meet  this  specifi- 
o  cation  to  the  3-sigma  limits  of  a  typical  normal  distribution  as 

shown  in  Figure  13..15(A),  there  are  only  3  out  of  1,000  circuits 
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which  require  parts  changes  t-  meet  the  specification. 

In  many  instances,  however,  designs  have  allowed  production  to 
meet  the  specification  only  to  the  1-sigma  limits,  as  shown  in 
Figure  13-15(B) ,  whence  100  tines  as  many  circuits  (31.7%) 
require  costly  parts  changes  involving  special  selection  to  meet 
the  specification.  This  results  in  a  nearly  rectangular  distri¬ 
bution.  With  this  distribution,  there  are,  of  course,  many  more 
circuits  near  the  speci ficat ion  limits  than  in  the  case  of  Figure 
13-15 (a) . 

If  severe  environments  and  operating  conditions  allow  only  equip¬ 
ments  using  those  circuits  measuring  between  96  and  192  volts  on 
the  production  line  to  "hit  the  target" ,  about  30%  of  the  equip¬ 
ments  using  the  circuits  of  Figure  13-15(b)  would  fail  to  do  their 
job,  compared  to  only  4%  failing  with  the  circuits  of  Figure  13- 15(A). 
Thus,  much  higher  reliability  results  in  the  latter  case,  even 
though  the  circuits  met  the  speci fication  in  both  cases.  From 
this  example,  it  is  easy  to  see  why  we  are  striving  for  designs 
to  meet  at  least  the  3-sigma  limits  for  their  tolerances  without 
rework,  both  from  production  and  reliability  standpoints. 

Let  us  now  see,  with  the  aid  of  the  following  examples,  how 
tolerances  combine  to  meet  the  foregoing  objectives. 

Example  1:  Series  Resistance  Tolerance;  When  parts  values 
having  a  normal  frequency  distribution  ar~  combined,  the 
resultant  value  will  exhibit  a  tolerance  advantage.  If  three 
such  resistors  (3,000  ohms,  2,000  ohms,  and  1,000  ohms)  each 
of  _+10%  tolerance  are  connected  in  series,  the  total 
resistance  expected  will  be: 

R„  *  6,000  +  /3002  +  2003+  100*=  6,000  t  37  4  ohms  -  6,000  t  6.2% 

This  will  be  the  combined  value  with  the  same  probability 
that  each  resistor  range  is  *10%?  that  is,  if  the  tolerance 
of  each  resistor  is  -10%  in  99.9%  of  the  cases,  the  sum  v/ill 
be  6,000  ±  374  ohms  in  99.9%  of  the  cases.  If  more  resistors 
are  combined,  rhe  over-all  tolerance  improvement  will  be 
greater. 

Combined  tolerance  =  ts  -  / tf+  t2  +  t?  ■+  .  ..t^ 

where  tj  ,  t2 ,  etc.,  are  the  individual  tolerances,  each  of 

which  must  contain  the  same  number  of  sigmas.  The  resulting  4 

combined  tolerance  will  also  contain  this  same  number  of 

standard  deviations.  That  is,  if 
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t,  -  A  a,  ,  t~  =  A  a2  ,  etc.  where  A  is  some  constant, 
then 

t  ~  Ac 
sum  sum 

Thus,  it  is  seen  that  the  equation  for  ts  above  is  derived 
from  the  more  basic  equation 


It  is  preferable,  wherever  possible,  to  express  a  result  as 
a  sum  of  values  to  utilize  the  mathematics  of  probability 
more  easily.  In  many  cases,  this  can  be  done  by  using  the 
logarithms  of  values  which  are  to  be  multiplied  or  divided 
and  by  using  the  reciprocal  of  values  which  combine  in  the 
same  manner  as  parallel  resistances. 

Example  2:  7-Stage  I-F  Amplifier  Gain  Variations .  Let  us 
check  a  7 -stage  i-f  amplifier  for  gain  limits,  where  each 
stage  uses  the  same  tube  type  with  a  bogey  transconductance 
of  5,000  microhms  and  a  1-kiiohm  composition  load  resistor 
(very  lightly  .loaded)  .  The  tube  and  resistor  variations  used 
are  those  given  in  reference  (8),  wherein  the  ±  values  are 
the  3-sigma  limits  for  normal  distributions  centered  about 
the  values  preceding  them. 

For  Lowest  Expected  Gain: 

Tube  C on t r i bu t i on : 

(gm  low  by  21%  *15%) 

gm  *  5,000  (0.7 9  t  0.15)  =  3,950  ±  750 

Gain  per  stage  (nominal  ra  )  =  9^!=  3.95  t  0.75  =  11.9  i  1.7  db 

Tube  and  Resistor  Contribution: 

rs  low  by  8.5%  t  6.7%  contributes  a  decrease  of  0.77  db 

i  0.64  db 

Then  gain  perstage  -•  11.9  db  -  0.77  db  i  1.7  db  i  0.64  db 
-  11.13  ±  ^1.72+  0.643  db 
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7-stage  gain  =  7  x  11.13  ±  .7(1.72+  0.642)db  =  77.9  ±  4.8  db 

For  Highest  Expected  Gain: 

Tube  Contribution: 

(gm  high  by  15%  -  15%) 

=  5,000  (3,15  ±  0.15)  =  5,750  ±  750 
Gain  perstage  (nominal  ra )  =  gmrj  =  5.75  ±  0.75  =  15.2  t  1.1  db 
Tube  and  Resistor  Contribution: 

high  by  12%  i  5.4%  contributes  an  increase  of  1  db  ±  0.4  db 
Then  gain  perstage  =  15.2  +  1  ±  1.1  i  0.4  db  -  16.2 
±  %/l .  1 2  t  0.42  db 

7-stage  gain  =  7  x  16.2  ±  J7 (l.l3  ±  0.42)db  =  113.4  ±  3.1  db 

Therefore,  the  gain  of  this  amplif  er  under  typical  production 
and  operating  conditions  is  expected  to  lie  between  73.1  db 
and  116.5  db.  During  the  past  several  years  in  which  i-f 
strips  have  been  in  production  at  Mctorola,  this  gain  varia¬ 
tion  has  been  shown. 

Through  use  of  these  principles,  other  mere  complex  circuits 
have  been  successfully  investigated  to  determine  whether  or 
not  they  had  adequate  safety  margins  to  meet  their  required 
tolerances.  This  approach  has  been  valuable  not  only  in 
avoiding  production  of  unreliable  equi’^ment,  but  also  in 
avoiding  the  wasted  breadboarding  of  circuits  which  are  in¬ 
capable  of  performing  consistently  within  required  limits. 

Reference  (8)  provides  some  electronic  part  variance  data  updated 
to  March  1963.  Some  parts  tolerance  dat  i  is  provided  in  chapter 
18. 

The  design  engineer  can  adjust  part  tolerances  (distributions) 
until  the  probability  of  acceptable  component  performance  is  at 
least  equal  to  required  ccxnponent  reliability.  Rigorous  such 
analysis  can  be  very  complex,  often  requiring  a  computer.  But 
it  leads  to  actually  needed  part  tolerances,  minimum  manufac¬ 
turing  cost,  and  required  reliability.  The  design  will  approach 
the  correct  reliability  needed  for  best  co3t-e f feet iveness ,  but 
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not  necessarily  the  hiqhest  possible  reliability. 

1.4.3  Marginal  Checking  has  beei.  developed  (reference  15)  to 
(a)  make  graphically  clear,  in  an  explicit  quantitative  way,  what 
tolerance  a  given  circuit  has  to  variations  in  its  components, 
and  (b)  provide  a  method,  usable  in  the  later  systems  phase,  of 
preventive  maintenance  that  will  adequately  coDe  with  the  pro¬ 
blems  of  preventive  maintenance  that  will  adequately  cope  with 
the  problems  of  component  deterioration.  Such  a  method  has  been 
extensively  used  in  the  design  phases  of  large  real-time  control 
systems,  as  well  as  in  day  by-day  operation  of  such  systems. 

This  section  discusses  the  use  cf  marginal  checking  in  the  design 
phase.  The  allowable  variation  of  a  component  is  determined  as 
a  fnnrt-  i  on  of  a  selected  circuit  parameter,  usually  a  supply 
voltage.  This  measures  the  margins  or  circuit  p*.-.  fc. .  m * n re  in 
terms  of  the  marginal-checking  parameter. 

In  practice,  the  tolerance  of  one  of  the  components  in  the 
circuit  is  plotted  against  the  variation  in  this  marginal-checking 
parameter,  as  illustrated  in  Figure  13-20.  The  intersection  of 
mean-value  and  normal  marginal-checking  parameter  lines  near  the 
center  of  the  parabola  indicates  the  operating  point  of  the 
circuit  -  normal  voltage  on  the  circuit  and  normal  value  of  the 
components.  By  considering  the  supply  voltage  as  the  marginal¬ 
checking  parameter  and  lowering  it,  a  point  is  plotted  on  the 
contour  line  where  the  circuit  fails  to  perform.  This  failure 
can  be  defined  as  the  poirt  at  which  the  function  of  the  circuit 
deviates  from  that  pret  „ ibed  in  the  specification.  In  an 
oscillator,  for  instance,  the  point  at  which  the  frequency  shifts 
out  of  tolerance  can  be  considered  failure;  in  a  flip-flop,  the 
point  at  which  some  standard  pulse  fails  to  switch  the  position 
may  be  failure. 

Changing  the  tolerance  on  the  component  by  jome  factor  such  as 
10  per  cent  marginal-checking  voltage  will  result  in  a  different 
failure  point,  such  as  Point  2  on  the  curve.  Raising  the  toler¬ 
ance  of  the  component  10  per  cent,  another  failure  point,  Point 
3,  can  be  plotted.  Continuing  this  study,  a  contour  represen¬ 
ting  the  locus  of  the  failure  point  of  the  circuit  to  tolerance 
in  componentry,  as  a  function  of  some  marginal-checking  parameter, 
can  be  drawn  enclosing  an  area  of  reliable  operation.  This  sort 
of  study  often  results  in  finding  that  the  contour  is  not  symme¬ 
trical  about  the  operating  point,  and  that  wide  safety  margins 
occur  on  one  side  but  very  narrow  margins  occur  on  the  other. 

It  is  interesting  to  note  that  such  contours  change  radically 
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with  thi.  type  of  circuit.  In  most  cases,  the  contour  would  be  a 
close'  loop  if  the  marginal-checking  parameters  could  be  varied 
far  enough  without  damaging  the  components-  It  y  =  probably 
evident  that  plotting  the  curves  and  varying  each  of  the  com¬ 
ponents  in  even  a  moderately  complex  circuit  represents  a  rather 
long  ar.d  tedious  study.  however  the  designer  can  hardly  afford 
to  be  ignorant  of  how  much  margin  a  circuit  has  before  it  will 
fail.  The  acceptability  of  the  circuit  to  the  system  car.  be 
based  only  on  such  knowledge. 

On  an  experimental  model  or  "breadboard"  the  effects  of  manufac¬ 
turing  variance,  cold  flow,  drift,  or  aging  can  be  simulated  by 
insertion  of  appropriate  spacers  or  resistors,  by  altering 
voltages  or  currents,  etc.  For  example  lowering  electron  tube 
heater  voltage  simulates  cathode  deterioration,  with  correspon¬ 
ding  effect  upon  transconductance.  Diode  forward  resistance  may 
be  simulated  by  adding  series  resistors.  Transistors  having 
tolerance  limit  values  can  be  substituted  to  study  the  effects. 
Brakes  can  ue  auuea  to  stu  Llit:  c!  I  ItiLl  Ot  wearout  that  leads  to 
fr  iction . 

To  the  extent  that  such  tests  can  be  conducted  under  true  opera¬ 
tional  environment  and  stress,  they  can  be  very  useful.  But  the 
analytical  approaches  are  usually  better  able  to  fully  account 
for  all  manufacturing,  environment,  stress,  and  time  factors,  as 
well  as  to  provide  derivative  insight,  and  they  are  usually  less 
co?  *■ 1  v . 

1.5  FAILURE  RATE  PREDICTION 

When  failure  rate  information  is  obtainable  for  any  of  the  parts 
or  components  comprising  the  tentative  design,  basic  reliable 
design  demands  that  it  be  obtained  and  used  as  a  tradeoff  con¬ 
sideration  in  each  design  decision. 

1.5.1  "Generic"  r>at.a  in  Design:  For  a  first  approximation  the 
sources  outlined  in  chapter  5  can  be  used  to  predict  the  designed 
component  failure  rate.  If  the  resulting  prediction  is  an  order 
of  magnitude  higher  (say  5-to-l  or  20-to-l)  than  the  required 
failure  rate,  then,  in  spite  of  the  data  variability,  the  design 
probab ly  will  have  excessive  failure  rate.  The  designer  then 
has  to  do  something  to  his  design  to  get  it  down,  and  the  list 
of  failure  rates  on  his  design  parts  list  will  indicate  where  to 
look  for  improvement. 

For  a  particular  design,  a  search  of  the  reliability  literature 
will  often  turn  up  better  data,  particularly  for  parts  and  com- 
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ponents  not  covered  by  the  above  sources. 

1.5.2  Source  Pat  a  for  Design:  Since  the  above  generic  data 
tells  the  design  engineer  very  little  about  the  failure  rates  of 
the  specific  parts  or  components  he  wa:  's  to  use,  he  must  get  it 
elsewhere.  Some  contractors  have  data  collection  systems  that 
collect  failure  mode,  operating  time  or  cvcles,  stress,  and  en¬ 
vironment  data  on  all  parts  and  components  they  use.  When  the 
accumulated  time  or  cycles  is  great  enough,  such  data  will  serve 
very  well. 

Moat  parts  and  component  manufacturers  doing  business  with  the 
military  have  new  had  to  collect  such  data  on  their  own  products, 
from  their  own  tests,  from  their  customers,  or  from  the  military 
users.  Therefore  a  prime  source  of  such  data  is  the  manufacturer. 
For  competitive  reasons  he  may  refuse  to  publish  it.,  but  it  is 
usually  obtainable  on  a  confidential  or  informal  basis,  not  to 
be  quotea.  Another  way  to  get  it  is  to  ask  for  a  quotation  or. 
delivery  of  a  number  of  parts  that  are  guaranteed  to  meet  the 
specified  reliability.  It  the  specified  value  is  not  realistic, 
one  will  quickly  find  out  what  is. 

Another  source  is  the  manufacturers  customers  who  may  also  be 
less  biased  and  more  willing  to  provide  it. 

In  any  event,  the  contractors  design  engineer  has  to  systematical! 
look  for  such  specific  data  among  such  sources.  It  will  not  be 
found  in  convenient  handbooks. 

1.5.3  Test  Data;  If  satisfactory  data  cannot  be  obtained  as 
above,  the  contractor  may  need  to  conduct  tests,  or  as*  the 
suppliers  to  do  it-  See  chapters  7  and  11.  On  the  other  hand 
the  reliability  requirement  may  be  so  hiqh  that  such  *^>sts  would 
be  too  costly  or  take  too  long. 

1.6  HUMAN  ENGINEERING 


All  system  and  hardware  designs  are  operated  by  people,  and 
people  make  mistakes.  Many  such  mistakes  result  in  failure  of 
the  system  to  perform  its  function.  Therefore  human  reliability 
is  just  as  important  to  system  reliability  as  hardware  reliabil¬ 
ity,  and  often  more  so.  The  reliability  of  people,  however,  can 
be  remarkably  influenced  by  the  design  engineer  in  many  ways, 
for  which  detailed  treatment  is  given  in  chapters  8  and  14. 
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1.7  FAILURE  CAUSE  &  EFFECT  AVO TD ANCE 


Chapter  12  provides  the  detailed  techniques  of  f’^iure  Modes  & 
Effects  analysis.  This  is  a  very  powerful  tool  that  works  both 
qualitatively  and  quantitatively.  It  provides  remarkable  visi¬ 
bility  to  the  design  engineer,  so  that  he  can  design  around 
potential  failures. 

Each  rime  a  failure  mode  and  its  effect  are  established  as  above, 
there  are  two  avenues  for  potential  reliability  improvement.  One 
is  to  examine  what  would  cause  the  particular  failure  mode,  and 
to  explore  the  possible  ways  that  the  design  can  be  altered  to 
reduce  or  eliminate  the  cause  without  causing  some  other  failure 
mode  or  effect.  It  works  more  frequently  than  might  be  imagined. 

The  other  avenue  is  to  examine  the  effects  of  each  failure  mode, 
and  to  explore  the  possible  ways  that  the  design  can  be  altered 
to  reduce  or  eliminate  adverse  effects  without  causing  some  ether 
failure  mode  or  effect.  This  is  the  more  commonly  stated  objec¬ 
tive  of  such  analysis. 

1.8  PREVENTIVE  MAINTENANCE 

Whenever  a  part  or  component  has  a  "wearout"  failure  rate 
characteristic,  meaning  that  after  some  period  of  operation  its 
failure  rate  begins  to  rise,  obviously  reliability  is  preserved 
by  timely  preventive  maintenance.  Examples  are  (al  friction 
interfaces  such  as  cams,  (b)  members  under  high  fatigue  stress, 

(c)  devices  exposed  to  corrosion,  etc. 

The  usual  experience  with  manufacturers  recommended  preventive 
maintenance  schedules,  both  in  and  outside  the  Navy,  is  that 
they  are  not  religiously  followed.  So  failures  occur.  Therefore 
if  reliability  is  a  prime  objective,  the  contractors  design 
engineer  must  make  every  effort  to  avoid  the  need  for  preventive 
maintenance.  Obviously  this  is  not  always  possible,  but  all  such 
items  must  be  included  as  failure  modes  in  chapter  12. 

Where  the  need  for  preventive  maintenance  cannot  be  avoided,  the 
design  should  provide  for  the  longest  possible  period  between 
such  maintenance,  and  above  all  must  be  consistent  with  the 
overall  maintenance  policy  of  section  4.9,  the  availability  of 
Skills,  and  accessibility  as  in  sect  ion  4.6. 

Finally,  the  technical  manuals  must  emphatically  call  out  the 
schedule  and  importance  of  such  maintenance  to  reliability,  and 
it's  a  good  idea  to  spot  prominent  labels  like  "Lube  with  xxx 
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every  30  days"  next  to  the  fitting  on  the  equipment  itself. 

Failure  to  adhere  to  preventive  maintenance  schedules  will  always 
reduce  reliability. 

1.9  PRODUC  IDIi/ITY 

It  is  often  alleged  by  contractors  manufacturing  departments, 
sometimes  with  an  element  of  truth,  that  "engineering  is  just 
trying  to  design  it  so  we  can't  make  it."  Or  what,  might  be 
worse,  "We'll  make  it  that  way  regardless  of  cost  or  consequences." 
The  conseouences  are  often  unreliability .  What  are  the  elements 
of  proaucibility? 

1.9.1  Procurability :  The  c  •’nones t  complaint  is  specification 
of  purchased  components  in  such  a  way  that  they  are  obtainable 
from  only  one  source.  This  places  the  procurement  people  at  the 
mercy  of  the  supplier  who  can  simply  say  "This  is  the  best  re¬ 
liability  I  can  provide,  but  I  can't  guarantee  what  it  is.  Take 
it  cr  leave  it."  Specifications  must  be  so  written  that  the 
supplier  knows  what  reliafcili  y  he  will  have  tc  prove  to  get 
paid,  and  knows  that  one  or  two  competitors  can  do  it  i £  he  does 
not.  Thus  the  reliability  spec?  r'ication  must  be  truly  achiev¬ 
able,  verifiable,  and  competitive. 

1.9.2  Manufacturability ;  A  good  design  engineer  must  know  the 
machine  tool  capabl'ity  of  his  factory,  and  the  standaid  parts 
with  which  it  has  experience.  Design  within  this  capability  and 
experience  permits  the  factory  to  very  closely  approach  the 
inherent  design  reliability. 

Con  ersely  design  requiring  complex  special  tooling,  special 
parts,  and  exotic  materials  with  which  the  factory  has  no 
experience  inevitably  leads  to  poor  reliability  while  the  factory 
learns  how  to  deal  with  them.  And  the  factory  will  not  have 
discovered  all  problems  prior  to  delivery  of  the  first  product. 

1.9.3  Testability ;  Freqr  _,.tly  a  design  is  such  that  it  cannot 
ue  adequately  tested  to  seme  vital  specification,  an  obvious 
opportunity  for  unreliability.  Design  review  (chapter  15)  must 
make  sure  that  the  expected  assembly  and  test  sequence  is  such 
thoU  every  specification  can  be  tested.  Of  course  this  is 
particularly  true  for  reliability  verification  tests  (chapter  11). 

1.10  SUPPLIER  EVALUATION  &  CONTROL 


The  contractors  design  engineer  is  completely  responsible  for 
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seeing  that  his  design  meats  specifications  in  every  respect. 
Since  a  large  part  of  most  designs  consists  of  components  pro¬ 
cured  from  suppliers,  part  of  his  job  is  seeing  that  they  too 
understand  and  can  comply  with  the  reliability  requirements. 

Most  contractors  supplier  survey  systems  evaluate  the  suppliers 
"reliability  program",  but  do  not  evaluate  the  supplier  design 
engineers  knowledge  of  reliability  technology  and  the  design 
steps  he  is  obliged  to  take  on  the  specific  design.  Therefore 
it  is  up  to  the  contractors  design  engineer,  with  the  assistance 
of  reliability  engineers,  to  satisfy  himself  that  the  delivered 
component  will  arrive  with  achieved  and  verified  reliability. 
There  is  no  substitute  for  personal  engineering  contact. 

Detailed  information  on  Supplier  Control  will  be  found  in 
chapter  li. 


2.  RELIABILITY  IMPROVEMENT 

When  "good"  design  practices  are  used,  and  the  above  basic 
reliable  design  techniques  are  used,  the  contractors  design 
engineer  is  often  faced  with  the  realization  that  his  design 
still  does  not  have  adequate  reliability.  This  section  lists 
the  "strong  measures",  usually  expensive  and  time  consuming, 
that  he  can  consider  next. 

When  the  design  is  as  simple  as  it  can  get,  and  its  parts  are  of 
the  highest  available  reliability,  but  the  predicted  component 
or  system  reliability  is  still  far  from  the  actual  requirement, 
what  to  do-’  Let's  first  list  some  things  not  to  do,  though  they 
are  quite  commonly  encountered: 

Do  not  let  the  contractor  ignore  the  requirements,  if  it  was  de¬ 
termined  carefully  via  cost-effectiveness  analysis  at  the  next 
and  higher  levels.  It's  just  as  important  as  a  horsepower  or 
voltage  requirement.  If  the  higher  reliability  is  in  fact  not 
achievable  (seldom  the  case)  then  the  analysis  might  show  that 
the  maintainability  requirement  must  be  changed. 

Do  not  let  the  contractor  tell  himself  (or  others)  that  "well 
I'm  very  experienced  in  this  field,  and  if  this  is  the  best  I 
can  do,  no  one  can  ask  for  more."  Someone  can  and  had  better, 
if  the  system  is  tc  work  as  planned. 


Do  not  let  the  contractor  raise  the  predicted  parts  reliability 
to  make  it  come  out  right,  gambling  that  the  parts  reliability 


will  be  letter  when  it's  built.  Experience  shows  that  typical 
MTBF  growth  is  only  about  20%  per  year. 

Do  not  ignore  the  potential  advantage  of  judicious  redundancy, 
whose  reliability  benefit  can  be  phenomenal  relative  to  the  cost 
and  weight,  if  any,  added. 

Before  embarking  on  the  following  program,  however,  the  designer 
should  seriously  ask  "Is  this  trip  necessary?"  He  should  go  back 
to  the  higher  level  source  of  the  reliability  specification, 
explaining  what  is  likely  to  be  involved,  to  find  out  whether 
the  specification  can  be  relaxed.  Often  as  not  it  may  be  found 
that  the  excess  failure  rate  can  be  absorbed  in  some  other  part 
of  the  system  that  now  appears  to  have  better  than  anticipated 
reliability.  Or  sometimes  he  may  find  that  the  specification 
was  not  so  firm,  after  all,  when  the  achievement  cost  and  time 
are  considered. 

2.1  EVALUATION  TESTS 

Perhaps  the  commonest  approach  to  reliability  improvement  is  the 
construction  of  one  or  more  models,  prototypes,  or  "breadboards" 
of  the  questionable  portion  of  the  design.  Or  the  procurement 
of  a  test  quantity  of  components. 

If  stress/strength  margin  is  the  primary  question,  stresses  are 
measured  under  simulated  load  and  environment,  and  strength 
obtained  by  testing  a  number  of  units  to  failure.  If  tolerance 
buildup  is  the  question,  a  systematic  worst-case  simulation  is 
conducted.  If  human  compatibility  is  the  question,  tests  are 
conducted  using  operators  or  maintenance  people.  If  failure 
effect  is  the  question,  failures  are  simulated.  If  produclbil- 
ity  is  the  question,  manufacture  and  assembly  of  the  models  will 
show  it. 

If  failure  rate  prediction  is  the  question,  it  may  or  may  not  be 
feasible  to  conduct  life  tests,  depending  upon  mission  time, 
quantity  cost,  etc.  "Accelerated”  tests  are  frequently  considered, 
but  unless  a  bona-fide  correlation  between  operational  and  over¬ 
stress  can  be  proven,  they  are  meaningless.  For  example  trans¬ 
istor  failure  rates  at  excessive  temperatures  can  b'  easily  cor¬ 
related  to  operational  temperature,  but  this  provides  little  or 
no  in  format ion  about  random  failure  rates  for  all  causes  at 
operational  temperature. 

In  all  the  above  tests,  the  objective  is  to  (a)  determine  pre¬ 
cisely  where  and  how  it  fails,  (b)  modify  the  design  to  avoid 
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the  cause  or  effect,  and  (c)  recycle  until  the  required  reliabil¬ 
ity  is  achieved  (see  chapter  11) . 

2.2  LOCAX  ENVIRONMENT  CONTROL 


Often  it  becomes  apparent  during  design  that  the  severe  environ¬ 
ment  is  about  all  that  prevents  achieving  the  required  reliabil¬ 
ity.  The  design  engineer  is  faced  with  a  choice  between  improv¬ 
ing  the  component  to  withstand  the  environment,  or  improving  the 
environment  to  satisfy  the  component.  Such  local  environment 
control  nearly  always  adds  weight,  space  and  cost,  so  he  has  to 
evaluate  the  tradeoff  on  the  basis  of  cost-effectiveness. 

Often  ignored  by  the  contractors  design  engineer  is  the  harmful 
effect  of  factory,  transportation  and  installation  environments, 
as  opposed  to  operational  environment.  Cross-country  trucking 
temperatures  can  get  very  high,  and  shock  levels  often  far  exceed 
the  operational  specification.  Obviously  improved  packaging  and 
special  handling  instructions  m<  ’  be  necessary  to  preserve  high 
reliability. 

Here  are  some  examples  of  local  environment  control. 

2  .2.1  Temperature :  Figure  13-28  reproduced  from  reference  (8) 
shows  the  generalized  effect  cf  temperature  on  the  failure  rate 
of  electronic  parts.  Similar  curves  are  available  for  many 
specific  electronic  parts,  and  can  be  generated  for  many  mechan¬ 
ical  parts.  Thus  to  improve  reliability  the  design  engineer  can 
consider  such  provisions  as  freer  convection,  radiation  fins, 
forced  air  or  water  cooling,  better  heat  source  distribution, 
reduction  of  heat  generation,  and  even  conventional  refrigera¬ 
tion.  Of  course  as  the  means  o^  temperature  control  becomes 
more  complex,  and  its  own  reliability  is  taken  into  account,  a 
point  of  diminishing  returns  can  be  reached. 

2.2.2  Humidity :  High-impedance  electronic  circuits  are 
particularly  sensitive  to  humidity,  but  low- impedance  transistor 
circuits  are  seldom  affected.  Corrosion  of  mechanical  and 
electrical  components  is  of  course  promoted  by  humidity.  Control 
can  be  effected  by  hermetic  sealing,  dessicants,  air  flow, 
heaters,  refrioeration,  etc. 

2.2.3  Vibration  &  Shock:  General  displacement  of  the  vacuum 
tube  by  semiconductors  has  greatly  improved  electronic  circuit 
reliability  in  vibration  environment.  Mechanisms  are  subject  to 
wearout  unless  designed  for  the  vibration.  Shock  mounting  can 
be  used  to  control  the  environment,  if  all  possible  excitation 
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frequencies  are  considered. 

2.2.4  Radiation?  Semiconductors  are  still  very  sensitive  to 
radiation,  the  effect  being  a  progressive  deterioration.  So  are 
people.  The  only  controls  are  very  heavy  and  bulky  shields. 

2.3  FAILURE  PREDICTION  DEVICES  : 

Sometimes  when  there  seems  no  feasible  way  to  improve  reliability 
of  the  components,  and  redundancy  does  not  improve  cost-effect¬ 
iveness,  ways  can  still  be  found  to  detect  an  approaching  failure 
in  time  to  head  it  off.  The  result  is  effective  reliability 
improvement.  Here  are  some  examples: 

2.3.1  Temperature :  We  are  all  familiar  with  the  widespread 
use  of  temperature  as  a  dependable  indicator  of  trouble  in 
diesel  engines,  people,  etc.  Thermometers  and  thermocouples  are 
designed  into  engine  systems  so  that  their  operators  can  monitor 
well-being  and  take  steps  to  keep  them  from  failing.  Contacts 
may  be  used  to  sound  or  flash  alarm.  Without  such  indicators 

on  manually-operated  or  maintained  systems,  they  would  be  much 
less  reliable,  and  their  failures  more  costly.  Therefore  the 
lesson  is  to  look  for  le.ss-conventional  places  where  temperature 
monitoring  can  achieve  still  higher  reliability. 

2.3.2  Sound :  Some  mechanical  systems  telegraph  incipient 
failure  to  anyone  attentive  by  increasing  their  operating  sound 
level  (16).  By  spotting  microphones  at  critical  places  and 
periodically  comparing  sound  level  to  a  standard,  one  can  record 
the  change  from  previous  readings.  If  a  previous  correlation 
has  been  established,  the  part  can  be  replaced  with  one  that 
does  not  complain  of  poor  health.  Such  provision  can  be  designed 
into  the  equipment  to  improve  reliability. 

2.3.3  Other  Indicators:  Similarly  the  design  engineer  should 
consider  whether  substantial  reliability  improvement  can  be 
achieved  by  monitoring  pressure,  humidity,  vibration,  etc. 

Even  time  monitoring,  for  wearout  failure  rate  characteristics, 
can  be  very  useful. 

2.4  COMPONENT  INTEGRATION 


The  simplification  techniques,  such  as  value  engineering, 
achieve  higher  reliability  via  simplification  using  readily 
available  components  and  materials.  Component  integration,  a 
close  cousin,  does  it  by  deliberate  multiple  ue 3  of  common 
pieces  in  such  a  way  that  the  number  of  mechanical  and  chemical 
interfaces  is  greatly  reduced. 


13-30 


Electronics :  The  prime  example  of  such  reliability  improvement 
is  integrated  circuits  wherein  transistors,  diodes,  resistors, 
and  capacitors  are  all  plated  or  evaporated  onto  an  insulating 
substrate,  drastically  reducing  the  number  of  soldered  or  welded 
connections.  The  following  is  from  reference  (17): 

The  military  effect  on  the  progress  of  integrated  circuits  has 
been  twofold.  First,  new  technology  has  developed,  some  through 
the  direct  subsidy  of  military  research  and  development,  and 
much  more  through  the  company-sponsored  research  stimulated  by 
this  support.  About  $100  million  of  R  &  D  expenditures  escalated 
from  an  initial  Government  expenditure  of  $2  million.  The  second 
effect  has  been  the  military  agencies'  interest  not  only  in  using 
integrated  circuits  but  also  in  providing  the  market  and  the 
motivation  for  suppliers  to  complete  the  development  and  establish 
the  production  capability  to  supply  this  waiting  market. 

Military  and  space  applications  accounted  for  essentially  the 
entire  integrated  circuits  market  last  year,  and  will  use  over 
95  percent  of  the  integrated  circuits  produced  this  year.  Even 
in  1970,  these  applications  may  well  be  using  as  high  a  propor¬ 
tion  as  c5  percent  of  the  circuits  produced. 

The  "Dick  Tracy  wrist  television  set"  characteristics  of  integrated 
circuits  are  widely  known,  and  there  are  tremendous  size  and 
weight  reductions  in  electronic  equipment  using  these  techniques. 

In  many  applications,  particularly  those  in  which  weight,  is 
critical,  these  reductions  are  very  important.  This  is  not  the 
only  attribute,  however,  that  motivated  military  agencies  to  use 
integrated  circuits. 

Reliability  is  the  most  important  single  factor.  We  nave  data 
on  two  operating  medium-sized  computers  that  use  integrated 
circuits.  The  first  is  the  Apollo  guidance  computer,  designed 
by  MIT  and  built  by  Raytheon.  It  has  accumulated  19  million 
operating  hours  on  its  integrated  circuits,  in  which  time  two 
failures  have  occurred  --  an  initial  failure,  and  the  other  a 
failure,  external  to  the  package,  that  was  caused  by  moving  the 
computer . 

The  second  system,  the  MAGIC  1,  an  airborne  computer  built  by 
the  AC  Spark  Plug  Computer  Division,  has  accumulated  15-1/4 
million  hours  with  two  failures.  Fairchild's  in-house  life-test 
program,  with  33  million  total  operating  hours,  has  had  a  total 
of  eight  failures;  of  these,  five  accumulated  during  the  first 
6-2/3  million  hours  and  only  three  occurred  on  more  recent  units 
during  the  last  26-1/3  million  hours. 
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These  data  are  not  extrapolated  from  accelerated  tests,  but  are 
actual,  observed  operational  failure  rates,  and  include  early 
production  units  in  some  cases.  Considering  the  complexity  of 
the  function  performed  by  these  circuits,  the  integrated  circuit 
equipment  today  is  ten  ii.res  more  reliable  than  its  discrete 
component  counterpart.  As  new  failure  modes  are  identified  and 
eliminated,  we  may  see  substantial  improvements  in  the  reliabil¬ 
ity  figures.  Extensive  studies  of  this  area  is  underway. 

Today's  integrated  circuit,  with  minor  exceptions,  is  just  as 
sensitive  to  nuclear  radiation  environments  as  were  yesterday’s 
transistor  equivalents.  In  some  military  and  space  applications, 
this  will  place  a  serious  limitation  on  integrated  circuits  that 
US'  conventional  transistors  for  the  active  elements. 

The  most  liberal  way  to  measure  integrated  circuit  cost  is  to 
neglect  development  expenditures  and  to  consider  the  total 
mission  --  which  includes  initial  cost,  maintenance  and  repair, 
spare  parts,  logistics,  and  delivery.  For  satellite  applications, 
with  their  premium  on  weight,  integrated  circuits  are  cheaper 
to  use  than  conventional  circuits. 

Prices  of  individual  transistors  supplied  to  military  contractors 
range  from  $3  to  $5  in  small  quantity.  In  quantities  of  50,000 
or  more,  unit  prices  vary  from  75  cents  to  $2,  depending  upon 
transistor  type.  Tight  screening  and  burn-in  for  higher  relia¬ 
bility  will  increase  these  prices. 

By  comparison,  i f  we  consider  only  the  transistors  in  an  integrated 
circuit,  typical  prices  are  about  $4  per  transistor  in  small 
quantities;  and  in  quant itites  of  over  50,000  prices  of  $1.50  to 
$1.75  are  average.  The  reason  for  this  lower  cost  is  that  the 
silicon  chip  size  of  a  typical  12-ti ans istor  circuit  can  be 
smaller  than  that  for  the  2N1613  transistor. 

Performance  is  another  factor,  and  there  are  large  areas  of 
electric  equipment  that  cannot  be  equipped  with  integrated  cir¬ 
cuits.  In  general,  the  same  limitations  apply  to  integrated 
circuits  and  transistors.  For  example,  we  cannot  replace  the 
magnetron  in  the  radar  set,  and  it  is  difficult  to  make  accurately 
tuned  circuits  in  integrated  form.  However,  many  of  the  inte¬ 
grated  circuit  limitations  are  being  overcome  rapidly. 

In  developing  any  new  technology,  schedule  slippages  are  expected. 
The  electronic  industry  has  a  bad  reputation  in  this  area.  There 
are  many  cases  where  component  manufacturers  have  committed  them¬ 
selves  to  a  delivery  schedule  for  integrated  circuits  and  have  not 
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net  the  d e ad  1  ine . 

But  as  the  range  of  circuits  available  as  off-the-shelf  items  is 
expanded,  the  desimers  and  manufacturers  for  the  military  market 
will  rind  standai j  components  much  more  compatible  with  require¬ 
ments-  And  as  the  components  industry  gains  experier.  _e  with 
integrated  circuits  of  special  design,  manufacturing  and  delivery 
schedules  will  be  met  on  time. 

Integrated  circuits  now  satisfy  many  of  the  military  and  space 
requirements  and  there  will  be  an  increasing  use  of  invegrated 
circuits  ;  military  systems.  Today,  the  advanced  Minutxnan, 
Apollo,  Phoenix,  and  all  new  military  digital  computers  use 
integrated  circuits  for  the  ma-jor  part  of  their  electronics 
systems.  With  higher  reliability,  lower  cost,  and  better  per¬ 
formance,  many  missions  once  cons  idered  too  imaginative  have 
become  or  are  becoming  both  feasible  and  practical. 

Hydraulic  Systems:  Roughly  the  same  principles  have  been  used  for 
many  years  for  automatic  transmiss ions  and  servos,  where  the 
multiplicity  of  cylinders,  valves,  pipes  and  connections  are 
replace.!  by  a  common  casting  and  far  fewer  parts.  Order  of 
magnitude  reduction  in  the  mechanical  and  chemical  (corrosion) 
interfaces  provides  significant  reliability  improvement. 

Mechanical  x  Structural  Design:  The  desiqn  engineer  should  look 
for  component  integrat i  n  opportunities  via  casting,  forging, 
moldin’.,  plat  i  no,  etc.  Since  the  obiective  is  reliability  im¬ 
provement  and.  i'”'  t o t a  1  cost  reduction,  rather  than  just  tradi¬ 
tional  mans  fact  >r  in  ,  cost  reduction,  the  design  engineer  must 
re-evaluate  old  rules  o f  thumb  for  such  decisions. 
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Rebund  >ncy  Is  tie  poor  .s  Lon  of  more  than  one  way  t  1  accomplish  a 
funct  i  'P.  to  pr-'t  ect  anainst  failure  of  the  primary  means.  We 
often  hoar  st  at  •.•merit  s  like  "leaver  use  redundancy  except  as  a 
last  res. vt '  >r  "Redundancy  is  poor  design",  etc.  While  such 
st  at  s  am  some!  imes  correct  ,  usually  they  are  r.ot .  There 

are  many  desiun  situations  where  deliberate  redundancy  provides 
better  rid  i.u:;  '  i'y  improvement  with  a  total  cost  reduction. 
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for  which  some  detailed  analyses  are  given  in  Chapter  3. 

2.5.1  Functional  Redundancy:  Whenever  it  is  feasible  to  satis¬ 
fy  ?  required  total  function  via  multiple  components  of  smaller 
capacity,  such  redundancy  of  the  smaller  components  may  get  much 
higher  reliability  without  significantly  increasing  total  cost 

or  weight.  Cons iderations  of  flexibility  of  operation,  ccnsum- 
ability,  or  sheer  feasibility  of  capacity,  often  lead  to  the 
same  conclusion. 

Examples  of  functional  redundancy  are  (a)  a  task  force  of  ships 
instead  of  one  very  large  ship;  (b)  copilot,  to  take  over  both 
jobs  in  emergency; *  (c)  the  shared-load  fire  control  system  shown 
in  Figure  13-34.  The  analysis  (1)  of  such  a  fi^e  control  system 
shows  that  the  greatest  Availability  (A)  improvement  per  unit 
Acquisition  Cost  is  obtainable  in  the  radar. 

2.5.2  Operational  Mode:  Many  operational  systems  involve  sub¬ 
systems  needed  to  perform  different  functions,  but  which  can 
pinchhit  for  each  other,  perhaps  with  *-educed  performance.  An 
example  is  radar  and  optical  range  and  direction  finding  equip¬ 
ment  (18) .  They  are  each  best  for  certain  applications,  but  can 
be  used  as  backup  for  each  other  for  limited  ranges.  At  a  higher 
level  one  kind  of  ship  can  back  up  another  in  case  of  failure  or 
damage  .  At  a  lower  level  a  double-reduct  ion  gear  may  be  pro¬ 
vided  (19)  to  connect  the  ships  gas  turbine  generator  for  direct 
mechanical  emergency  propulsion. 

2.5.3  Override:  One  man  can  monitor  the  action  of  many  com¬ 
ponents  and,  if  they  are  designed  for  this  capability,  he  can 
"override"  or  compensate  for  component  failure.  This  form  of 
redundancy  is  so  extremely  common  that  we  may  not  so  recognize 
it.  Examples  are  automatic  pilot  override power  steering 
mechanical  override.  Even  competent  management  provides  such 
override  redundancy  to  compensate  for  sube'dinate  failure.  Less 
common  is  provision  of  automatic  mechanical . rather  than  human, 
override.  But  all  poss ib’ 1  it ies  should  be  weighed. 

2.5.4  stressed  Redundancy:  When  a  particular  component  (or 
part,  subsystem,  or  system)  is  known  to  be  relatively  unreliable, 
and  no  better  component  is  obtainable  for  the  required  function, 
the  design  engineer  may  consider  using  two  components  in  such  a 
way  that  they  are  both  operating,  and  therefore  stressed.  Often 
it  is  called  "parallel"  redundancy,  but  this  te-m  gets  confused 
with  parallel -vs -series  c  r.  f  igur  at  ion  . 


Thus  two  pumps  may  be  operated  in  parallel,  either  one  cf  which 
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can  handle  the  load.  Or  tv,,  norma  11  v- open  valves  in  series, 
where  shut-off  capability  must  be  assured .  Or  foui  resistors  in 
series -parallel  or  in  "quad".  Such  redundancy  obv iO”s ly  adds 
weight  and  cost,  and  cannot  provide  the  reliability  rains  achiev¬ 
able  thru  Sequential  Redundancy  described  below,  but  has  the 
distinct  advantage  of  avoiding  the  potential  unrei i abi 1 : ty  of 
automatic  or  manual  switching  to  "spares". 

In  a  broad  sense,  stress  strength  design  (section  2.4)  to  a  high 
reliability  margin,  involving  design  using  more  material  than  is 
necessary  to  handle  the  aver  age  stress,  is  a  form  of  stressed 
redundancy.  But  the  dos ion  engineer  must  realize  that  it  is  only 
one  of  many  such  redundancy  alternatives ,  and  select  the  most 
cGSt-e  f feet ive . 

2.5.5  Sequent lal  Redundancy  is  the  provision  of  spare  components 
in  such  a  way  that  they  are  not  stressed  until  place  into  service, 
so  that  longer  effective  life  can  be  expected,  often  this  is 
called  "standby"  redundancy,  which  too  easily  gets  confused  with 
standby  modes  of  operation  in  which  many  components ,  sometimes 
all,  are  .tressed. 

The  effective  reliability  gain,  assuming  instant  switching  with 
100%  reliability,  can  be  significant  (20'  as  shown  in  Figure 
13-36.  Note  that  the  potential  gain  detfiiorat.es  as  mission 
time  approaches  the  s i ng le- component  MTBF.  And  switching  is 
never  instant  nor  lOO1'  reliable. 

Examples  of  sequential  redundancy  are  legion.  Standby  pumps  are 
commonly  provided,  not  in  operation,  to  back  up  the  operating 
pump.  Critical  radio  receivers  are  o t ten  backed  up  with  duplicate 
receivers  automatically  switched  on  when  output  (noise  or  signal) 
fails.  Airport  tower  operators  are  backed  up  with  spare  opera¬ 
tors  who  only  follow  the  act  ion. 

In  the  broad  sense  corrective  maintenance  is  actually  sequential 
redundancy,  with  much  longer  time  constants,  and  may  be  analysed 
with  the  same  techniques .  But  again  it  is  the  design  engineers 
-job  to  determine  what  balance  o{  such  redundancy  alternatives  is 
the  most  effective. 

2.5.6  Redundancy  Le^el:  while  not  a  kind"  of  redundancy,  the 
level  at  which  redundancy  vs  used  has  much  impact  upon  the  weight, 
cost,  and  feasibility  of  achieving  a  given  high  reliability. 
Consider  a  critical  small  part  whose  failure  rate  is  1000  fail¬ 
ures  per  million  hours.  Since  it  is  critical,  it  contribute* 
1000/10'  hours  to  the  failure  rate  or  the  component  of  which  it 
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is  a  part,  and  to  the  next  level  subsystem,  system,  and  opera¬ 
tional  system  failure  rates.  Design  action  to  mitigate  or  com¬ 
pensate  for  it  can  be  taken  at  any  level.  And  sometimes  it  is 
much  easier  or  more  cost-effective  to  compensate  at  some  other 
level. 

Moreover  the  effective  MTBF  is  vitally  affected  by  the  number  of 
"modules"  into  which  the  system  can  be  divided,  assuming  a  given 
number  of  "spares"  is  ready  for  each  of  all  modules  (21).  Figure  13- 
38  shows  that  lower-level  redundancy  (more  modules)  is  more  effec¬ 
tive,  but  not  necessarily  the  most  economical.  One  can  divide 
the  relative  MTBF  values  by  number  of  spares  (say  70/5  =  14, 

or  25/3  =  8.3)  to  see  that  order  of  magnitude  reliability  im¬ 
provement  per  "spare"  is  achievable. 

Often  redundancy  is  achieved  at  the  system  level, -such  as  the 
multiplexing  of  two  large  computers  (22),  or  backup  of  a  vehicle 
with  another  complete  vehicle.  For  complex  systems  it  is  then 
often  found  that  the  automatic  switching  systems  become  so  complex 
(and  design  is  unproven)  that  their  poor  reliability  prevents 
significant  improvement  of  overall  reliability. 

2.5.7  Parts  Redundancy  Configuration?  Hundreds  of  reference* 
are  available  analysing  the  reliability  of  parallel,  series, 
series-parallel,  quad,  and  other  configurations,  for  "open"  and 
"short"  circuit.  Many  such  arrangements  are  commonly  used  in 
design,  the  "quad"  being  favored  recently  (23)  .  The  same  princi¬ 
ples  apply  to  higher  levels,  but  then  complexity  makes  them  diffi¬ 
cult  to  apply.  Such  analyses  will  be  found  in  Chapter  5. 

The  quad  configuration  uses  four  components  in  series  parallel,  as 
in  the  example  in  Chapter  5,  Figure  5-28.  If  the  component  were 
a  valve,  failure  of  one  valve  in  the  "short"  mode  (failure  to 
stop  the  flow  when  required)  would  not  cause  system  failure, 
since  the  other  three  would  effectively  stop  the  flow.  Likewise 
failure  of  one  valve  in  the  "open"  mode  (failure  to  permit  flow) 
would  not  cause  system  failure  since  the  opening  of  the  other 
three  valves  would  permit  flow.  The  quad  combination,  so  used, 
protects  against  single  failure  in  either  mode. 

2.5.8  Consequences  of  Redundancy:  The  major  disadvantages  of 
using  redundancy  to  solve  a  reliability  problem  are  weight,  cost 
and  complexity.  Usually,  (but  not  always)  providing  back  up 
systems,  parts  or  components  adds  the  weight  and  cost  of  the 
added  components.  Usually  the  added  weight  and  cost  is  reduced 
by  application  of  the  redundancy  to  the  smaller  sub -categories 
of  the  systems  (parts  rather  than  assembly) .  A  more  insidious 
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effect  may  be  increased  complexity,  which  may  easily  negate  in 
some  instances  the  reliability  improvement  sought.  For  example, 
where  a  back-up  system  or  component  is  energized  upon  the 
failure  of  the  primary,  the  addition  of  sensing  and  activation 
circuitry  or  mechanism  may  reduce  the  overall  reliability  below 
the  reliability  of  the  primary  system. 

Again,  where  duplication  of  equipment  is  provided  to  improve 
reliability,  the  cost  of  corrective  maintenance  is  not  necessarily 
"hanged.  The  cost  of  preventive  maintenance  may  be  essentially 
doubled . 

2.6  par. s  improvement 

Once  it  seemed  obvious  that  since  system  failures  are  caused  by 
parts  failures,  then  the  logical  course  would  be  to  make  the 
parts  more  reliable.  Several  years  and  $^0  million  later  we  get 
the  MTBF  of  only  a  dozen  or  so  Minuteman  electronic  parts  up  by 
an  order  of  magnitude.  As  General  James  R.  Bridges  said  (24), 

"I  seriously  doubt  that  we  will  ever  fully  meet  reliability 
requirements  for  space  systems  via  the  route  of  conventional 
electronic  parts  improvement" . 

If  the  MTBF  of  all  parts  of  a  component  could  be  raised  by  10-to-l 
(at  great  cost)  so  would  the  component  MTBF  be  improved  10-to-l. 
But  many  applications  demand  100-to-l  or  even  1000-to-l  to  become 
adequately  effective. 

But  it  is  one  very  important  method  of  achieving  higher  relia¬ 
bility.  The  Minuteman  parts  program  involves  elaborate  tolerance 
control,  unprecedented  engineering  and  manufacturing  controls, 
extensive  and  costly  testing  and  documentation  programs,  detailed 
traceability  identification,  and  special  packaging  and  handling 
procedures . 

The  contractors  design  engineer  must  therefore  look  for  opportu¬ 
nities  for  specific  parts  improvement  achievable  at  reasonable 
cost,  or  where  the  Minuteman  cost  levels  may  be  justifiable. 

3.  MAINTAINABILITY  DESIGN 


Like  reliability,  the  quantitative  amount  Ol  ainability 

needed  in  a  design  is  determined  by  cost-effectiveness  tradeoff 
analyses.  Once  it  is  determined  that  the  equipment  must  be  re- 
storable  to  operation  within  a  given  average  or  maximum  time, 
the  problem  then  is  how  to  design  to  achieve  it.  There  are  at 
least,  eleven  approaches  that  may  be  taken  depending  upon  the 
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specific  circumstances,  and  further  details  are  given  in  Chapter 
8.  (See  Reference  25). 

3.1  SIMPLIFICATION 

Simpler  design  are  nearly  always  easier  to  maintain  with  lesser 
skills  in  less  time.  See  section  2.1. 

3.2  STANDARDIZED  DESIGN 

As  discussed  section  2.2  of  this  chapter,  standardization  adds 
to  the  experience  with  a  specific  design  and  its  maintainability, 
thus  contributing  to  maintainability.  Interchangeability  further 
improves  maintainability. 

3.3  MODULAR  DESIGN 

Design  in  sets  of  standard  subassemblies  or  modules  permits  rapid 
standardized  diagnosis  and  replacement  thereof,  so  that  operation 
can  proceed  while  corrective  maintenance  is  done  on  the  bench. 

Or  the  module  may  be  designed  for  discard  upon  failure.  On  the 
other  hand  modular  design  usually  adds  electrical  or  other 
connections,  which  degrades  reliability  and  adds  acquisition 
cost,  so  a  tradeoff  analysis  is  necessary. 

3.4  ADJUSTMENTS 


A  design  with  the  fewest  possible  needs  for  adjustment,  alignment, 
or  calibration  improves  maintainability  by  reducing  the  required 
restoration  time  and  skill  level. 

3 .5  FAILURE  EFFECT  PROVISION 

Basic  to  all  design  for  maintainability  is  the  detailed  study  of 
failure  modes  (conducted  for  reliability  design  and  analysis) 
and  careful  provision  for  maintenance  resulting  from  such  failure 
e  f fects . 

3.6  ACCESSIBILITY 

The  word  is  often  considered  synonymous  with  Maintainability,  but 
is  only  one  indispensable  contributor.  Obviously  "human  engineer¬ 
ing"  principles  must  be  used  to  provide  good  accessibility  to 
critical  components  that  may  fail.  Perhaps  the  commonest 
deficiency  is  design,  so  that  one  or  more  other  components  must 
be  removed  to  get  at  the  one  that  failed. 
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3.7  SAFETY 


Maintainability  is  improved  by  designing  so  that  inadvertent 
damage  to  one  component  cannot  occur  while  working  on  another, 
or  by  improper  installation.  A  connector  must  only  fit  its  mate, 
among  those  handy. 

3.8  EVALUATION  TESTS 


Just  as  for  reliability,  many  situations  cannot  be  evaluated 
adequately  via  paper  design  and  analysis.  If  the  component 
maintainability  is  critical  (failure  to  restore  in  time  fails 
the  mission)  then  mockup  tests  are  imperative,  with  appropriate 
design  to  resolve  the  problems  thus  brought  to  light.  Fortunately 
such  tests  are  much  easier  and  less  costly  than  reliability 
evaluation  tests. 

3.9  IDENTIFICATION 

Much  downtime  is  contributed  by  inadequate  identification  of 
original  components  and  their  replacements,  particularly  when 
replacements  come  from  different  suppliers,  with  a  different 
number,  and  look  different. 

3.10  TOTAL  MAINTENANCE  POLICY 


The  restoration  time  for  any  one  component  is  of  course  dependent 
upon  logistic  availability  of  a  replacement  or  tools  to  repair, 
and  upon  adequate  skill  availability.  Conversely  the  total  system 
maintainability  depends  upon  the  integrated  summation  (not 
arithmetic)  of  all  component  maintainabilities.  Thus  maximum 
system  maintainability  can  be  achieved  only  by  considering  all 
tradeoffs  and  establishing  a  total  maintenance  policy  for  the 
system  design,  but  consistent  with  available  or  achievable 
maintenance  resources. 

3.11  FAILURE  DETECTION  &  ISOLATION  DEVICES 

Many  systems  and  components  can  fail  in  such  a  manner  that  the 
failure  is  not  apparent  until  a  later  time  when  its  consequences 
show  up.  Computers  can  thus  make  costly  mistakes.  An  oil  port 
to  one  bearing  can  become  clogged.  Thus  maintainability  of  such 
systems  can  be  helped  by  adding  critical  failure  detection  devices, 
such  as oomputa:  check  routines,  or  limit-contact  thermometers  for 
bearings.  For  complex  systems  such  devices  to  catch  part  failures 
would  be  prohibitively  expensive,  so  the  detection  is  done  at 
higher  levels.  But  this  introduces  the  need  for  failure  isolation 
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devices  which,  once  a  failure  is  known  to  have  occurred,  help  to 

locate  it  precisely.  These  all  contribute  to  desiqn  maintain¬ 
ability. 
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Chapter  14 
HUMAN  FACTORS 

A  basic  assumption  of  the  system  development  point  of  view  is 
that  man  can  be  considered  as  one  of  the  major  components  of  a 
total  system.  This  view  is  opposed  to  the  notion  that  man  simply 
plans,  buys,  develops,  and  uses  a  system  once  it  is  built.  Any 
reasonably  complex  system  requires  a  true  interaction  between 
man  and  the  other  parts  of  the  system,  which  may  be  machines,  other 
men,  or  combinations  of  these.  Some  way  must  therefore  be  found 
for  thinking  about  the  functions  of  machines  and  the  functions 
of  men  within  a  framework  which  makes  possible  the  relation  of 
these  two  kinds  of  functions  to  common  goals  --  that  is,  to 
system  goals.  Even  in  a  system  as  familiar  and  as  relatively 
simple  as  the  automobile,  it  is  easy  t.o  see  that  the  goal  of 
transporting  passengers  over  roads  requires  not  only  the  func¬ 
tions  of  the  machine  itself  but  also  a  considerable  variety  of 
human  functions  performed  by  the  operator,  as  well  as  auxiliary 
functions  performed  by  such  people  as  traffic  policemen  and 
filling-station  attendants.  The  design  of  a  system  which  is  to 
be  successful  in  achieving  seme  socially  defined  purpose  requires 
thorough  and  continued  consideration  of  the  interacting  functions 
of  both  men  and  machines. 

In  this  chapter  we  will  attempt  to  develop  three  basic  areas. 

(a)  The  proper  use  of  man  in  the  system,  based  on  his  capabil¬ 
ities  and  deficiencies;  (b)  The  unknown,  but  potent,  effect  on 
the  reliability  of  the  system  of  using  man  as  a  subsystem  or 
component;  (cl  Methods  of  integrating  man  into  the  system. 

The  " Human  Engineering"  aspects  of  human  factors,  dealing  with 
design  oi  hardware  for  compatibility  with  people,  is  not  covered 
in  this  chapter.  The  criteria  for  design  and  analysis  of  such 
"operability"  and  (human)  "maintainability"  will  be  found  in 
chapter  9,  a ince  it  is  often  associated  with  hardware  maintain¬ 
ability.  But  the  identical  principles  apply  to  design  for  oper¬ 
ability.  Chapter  8  also  references  the  principal  handbooks  on 
Human  Engineering,  which  has  a  very  significant  impact  on  relia¬ 
bility,  but  is  beyond  the  scope  of  this  course. 

1 .  MAN  AS  AN  ELEMENT 

What  we  propose  to  do  in  this  chapter  is  to  describe  some  of  the 
functions  of  man  and  to  show  bow  these  can  be  related  to  the 
functions  of  the  machine  environment  in  which  man  _s  placed  as 
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part  of  a  system.  We  shall  attempt  to  do  this  by  developing  and 
using  a  language  eh at  relates  i nput  for  the  human  being  to  his 
output ,  which  in  turn  becomes  an  input  to  some  other  portion  of 
the  system.  In  other  words,  in  dealing  with  man's  functions ,  we 
shall  be  identifying  the  k.nds  of  transformation  wnich  an  input 
under '-iocs  in  order  to  be  effected  as  a  human  output . 

1.1  KTRPOSE  OF  MAN  IN  THE  SYSTEM 


A  system  is  developed  to  fulfill  some  human  purpose  or  intended 
use.  Its  purpose  may  be  to  protect  against  enemy  military  attack 
'V  to  harass  or  destroy  an  enemy  in  wartime.  But  systems,  of 
course,  are  not  confined  to  military  enterprises.  They  may  have 
distinctly  civilian  social  purposes,  such  as  those  of  an  airport- 
to-city  transportation  system,  a  mail-sorting  system,  a  check  - 
cas.ting  system.  Any  system  is  defined  in  terms  of  its  purpose. 

In  order  to  fulfill  a  purpose,  a  system  must  meet  certain  stan¬ 
dards  .  System  developers  have  been  known  to  take  the  point  of 
view  that  if  only  the  hardware  subsystem  can  be  made  to  run 
(perhaps  in  i  specially  prepared  test  location),  somehow  human 
beings  with  the  proper  characteristics  will  be  found  and  "fitted 
into"  the  system.  Such  a  view  places  too  much  dependence  on  the 
range  of  human  talents  and  on  the  availability  of  suitable  man¬ 
power,  as  well  as  on  the  extent  of  human  adaptability.  On  many 
occasions  this  restricted  view  of  systems,  and  system  develop¬ 
ment  has  led  to  failures,  breakdowns ,  costly  programs  of  retro¬ 
fitting,  and  even  to  virtual  system  abandonment.  No  system  is 
complete  "rit  i  1  it  can  be  shown  to  operate  within  a  total  setting 
that  includes  human  beings ;  no  system  r-an  truly  be  said  to  be 
-nc.ee  ess  tul  until  its  operational  effectiveness  is  dec  nstrated. 
The  best  system  he- -e  lopv.ent  is  that  which  includes  on.-  rder at  ion 
of  system  opera*  i  n  (rather  then  merely  hardware  op-'rabi  lity) 
from  the  very  beg  inning  of  system  design. 


Tn  one  concept  the  human  operator  is  pictured  as  a  "data  trans¬ 
mission  and  gi crossing"  link,  inserted  between  the  displays  and 
controls  of  a  machine.  An  input  is  transformed  by  certain  mech 
an  isms  into  a  signal,  which  is  displayed  as  a  pointer  reading, 
pattern  of  lights ,  an  oscilloscope  wave  form,  or  the  like.  Thi 
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an  isms  into  a  signal,  which  is  displayed  as  a  pointer  reading,  a 
pat  tern  of  lights ,  an  oscilloscope  wave  form,  or  the  like.  This 
i a  t  or  mat ion  is  read  by  the  human  operator  and  transformed  into 
j  es ponses  --  the  pushing  of  switches,  the  moving  of  control 
handles,  and.  op .  These  in  turn  generate  control  signals  which 
an.  t t  ans  termed  by  mechanisms  into  system  outputs . 
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functioning  enters  into  complex  systems  at  many  points  and 
ny  particular  ways.  Furthermore,  the  display  of  information 
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the  controls  to  which  the  individual  responds,  and  the  irechanisms 
which  provide  the  trans formations  for  these  components  of  the 
system  are  of  considerable  variety.  Accordingly,  we  need  to 
recognize  at  once  that  the  generalized  picture,  while  it  indicates 
man's  position  as  a  system  component,  does  not  provide  the  means 
for  a  detailed  analysis  of  the  variety  of  human  functions.  It 
would  be  a  mistake  to  think  that  because  mar.  typically  "occupies 
a  space"  between  machine  displays  and  controls,  his  functioning 
can  be  related  in  a  constant  set  of  ways  to  such  inputs  ar.d  out¬ 
puts.  The  fact  is,  neither  the  input  nor  the  output  by  them¬ 
selves  will  tell  us  the  nature  of  man's  functioning.  For  there 
are  different  kinds  of  transformations  which  may  be  performed 
(by  the  human  nervous  system)  in  turning  inputs  into  outputs. 

1.2  FUNCTIONS  PERFORMED 

Suppose  that  a  system  has  available  an  oscilloscope  as  a  basic 
unit  for  display.  On  the  face  of  this  scope  appears  a  60-cvcle 
wave  form  which  can  be  adjusted  in  amplitude  to  a  particular 
size,  the  required  size  being  given  by  two  fine  horizontal  lines 
on  a  transparent  overlay  placed  against  the  tube  face.  Let  us 
assume  that  the  system  requires  the  amplitude  of  the  wave  to  be 
determined  within  very  close  tolerances  before  further  -Deration 
of  the  system  can  take  place.  Following  this,  what  happens  is 
that  an  external  signal  distorts  the  form  of  the  wave,  and  it.  is 
the  human  operator's  task  to  "report"  the  nature  of  these 
distortions  by  pushing  one  of  five  buttons.  If  there  is  an 
amplitude  distortion  (vertical  displacement  from  the  overlay 
markers),  he  pusnes  button  1;  if  there  is  a  .requency  distortion 
(horizontal  displacement  from  other  markers),  he  pushes  button  2; 
if  there  are  additional  frequencies  present  (irregular  wave 
pattern),  he  pushes  button  ;  and  so  on. 

Now,  saving  certain  details  for  later  cons ideretion ,  let  us 
consider  the  difference  between  what  the  operator  does  in  "getting 
the  equipment  rea  \y  to  operate"  and  what  he  does  in  "operating." 
Actually,  he  is  utilizing  two  different  functions  in  the  two 
cases,  even  though  the  display  may  be  the  same  in  both. 

In  placing  the  equipment  into  proper  operating  condition,  the 
human  operator  is  making  use  of  the  function  of  sensing.  That 
is  to- say,  he  is  using  his  visual  receptors,  nervous  system,  and 
effectors  simply  to  "report"  (to  a  machine,  typically)  ^he 
presence  or  absence  of  a  difference  in  physical  energy.  In  this 
case,  the  physical  difference  being  reported  is  the  coincidence 
of  two  points  (o_  small  areas)  each  of  which  lies  along  a  narrow 
band  of  light  (a  "line")  which  makes  an  abrupt  gradient  of 
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intensity  with  its  surroundings .  The  operator  is  exercising  a 
function  called  visual  acuity,  a  particular  name  for  one  of  his 
sensing  functions 

it  may  be  noted,  however,  that  the  operator  is  able  to  do  much 
more  than  this,  even  within  the  equipment-readying  stage  of 
operation  we  are  considering.  He  is  perfectly  capable,  for 
example,  of  making  an  output  which  reports  the  amplitude  of  the 
wave  along  some  scale  such  as  millimeters  or  volts.  He  is  able 
t  tell  us  the  color  of  the  wave  form,  to  estimate  whether  it 
is  bright  enough,  or  whether  it  has  a  regular  appearance,  ar.d 
many  other  things.  Whv  does  he  not  do  all  t>.Bse  thingn  in  this 
situation?  There  is  no  mystery  to  this  question  at  alls  he 
does  not  simply  because  we  have  not  told  him  to  (or  perhaps  told 
him  not  to) ,  But  this  means  we  must  recognice  that  there  is 
more  to  the  matter  of  input  than  simply  the  presence  of  a  display. 
In  o r d  er  to  get  the  output  required  by  the  system,  the  operator 
must  be  provided  with  a  set  of  instr  actions . 

One  basic  purpose  ot  „hese  instructions,  we  are  now  able  to  see, 
is  to  determine  which  functions  "higher  up"  than  sensing  are  to 
be  shunted  out .  The  combination  of  the  oscilloscope  display  and 
the  instructions  is  what  determines  the  output  that  will  be  made. 
The  instructions  say  to  the  operator,  in  effect:  "Report 
coincidence  between  a  set  of  lines.  Do  not  report  their  shape, 
or  size,  or  brightness,  or  regularity,  or  meaning,  or  anything 
else."  Thus,  it  is  apparent  that  the  effect  of  presentation  of 
the  oscilloscope  display  plus  a  particular  set  of  instructions 
is  to  put  into  operation  a  particular  kind  of  human  function, 
sensing ,  and  to  shunt  out  other  Vinds  of  functions  of  which  the 
human  operator  is  capable. 

Now  let  us  contrast  this  lementary  kind  of  behavior  with  what 
occurs  when  the  equipment  is  being  operated  rather  than  merely 
turned  on.  In  this  case  the  human  operator  must  function  ..n 
quite  a  different  way.  He  must  provide  five  different  output 
responses  (press  one  of  five  buttons)  whenever  a  particular  kind 
of  deformation  of  the  sine  wave  appears,  whether  it  is  a  change 
in  the  horizontal  dimension  produced  by  variation  in  frequency, 
in  the  vertical  dimension  (amplitude) ,  or  in  one  of  several 
other  types.  In  other  words  he  must  identify  five  different 
classes  of  patterns  appearing  on  the  scope. 

When  the  operator  is  engaged  in  this  function  cf  identifying, 
has  the  internal  mechanism  for  sensi  tg  been  shunted  out?  Of 
course  it  has  not,  because  the  physical  differences  which  deter¬ 
mine  the  existence  of  classtv.  of  stimuli  to  which  the  operator 
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responds  must  be  sensed  in  order  for  identification  to  take 
place.  Thus  we  see  that,  on  the  input  end,  human  functions  have 
a  hierarchical  arrangement.  The  use  of  a  function  like  identi¬ 
fying  requires  that  a  function  lower  in  the  hierarchy,  sensing , 
be  put  in  operation  as  well. 

Again,  however,  we  can  see  that  certain  even  higher  functions 
have  indeed  been  shunted  out  when  we  ask  for  identification.  For 
example,  the  operator  may  be  capable  of  telling  us  that  a  deforma¬ 
tion  of  the  sine  wave  in  the  vertical  dimension  "means"  a  change 
in  amplitude.  But  we  have  not  asked  him  that;  we  merely  asked 
him  to  press  a  button  indicating  the  presence  of  a  particular 
class  of  change  in  wave  shape.  Or  again,  he  may  be  capable  of 
interpreting  this  kind  of  change  as  indicating  the  presence  of  a 
type  of  remote  signal  received  at  the  other  end  of  the  system. 
Again  we  see  that  one  of  the  primary  effects  of  instructions  is 
to  keep  the  human  being  functioning  at  the  proper  level  and  to 
shunt  out  other,  higher- level  functions. 

Another  internal  mechanism  must  be  added  at  this  level,  too,  and 
that  is  memory.  For  we  know  that  without  some  kind  of  long-term 
storage  of  representations  of  the  five  different  changes  in  wave 
shapes,  the  achievement  of  five  different  outputs  would  be 
impossible.  Instructions  alone  will  not  do  the  job.  To  be  sure, 
we  can  describe  to  the  individual  what  he  is  expected  to  do,  by 
means  of  instructions,  but  he  will  nevertheless  not  be  able  to 
do  it  by  this  means  alone.  He  must  have  an  internal  means  of 
matching  the  external  display  to  one  of  five  classes  in  order 
that  he  can  make  five  different  responses,  as  required  in  our 
example.  This  means  he  must  have  previously  acquired  the 
"representative  shapes"  in  his  memory,  by  means  of  learning 
preceding  the  occasion  when  he  tackles  the  job  of  operating  his 
equipment.  And  this  provides  the  basic  reason  for  training ,  as 
well  as  for  the  crucial  part  it  plays  in  the  system  development 
process „ 

Now,  this  description  of  two  examples  of  human  functioning  has  not 
involved  very  high-powered  psychology;  we  are  well  aware  of  that. 
We  have,  in  fact,  been  describing  the  functions  of  sens ing  the 
perceiving ,  which  have  been  studies  by  psychologists  for  many 
years.  But  the  purpose  of  our  account  has  not  been  to  review 
basic  principles.  Rather,  it  has  been  to  show  that  the  funda¬ 
mental  operations  in  describing  the  functions  of  a  man's 
behavior  are  the  operations  of  a  design  engineer  in  describing  a 
machine.  Psychologists  make  the  same  kinds  of  inferences  about 
human  behavior  as  designers  do  of  machines,  and  they  are  based 
upon  the  same  kinds  of  objectively  defined  operations.  It  should 
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therefore  be  quite  easy  for  the  designer  to  understand  the  nature 
of  human  functions,  provi  'ed  he  1  earns  what,  input  conditions 
must  be  met  and  what  the  output  achieves  (as  an  input  to  the  next 
unit  of  the  system).  In  our  further  delineation  of  human  functions, 
we  shall  find  it  useful  to  refer  back  to  the  conceptions  developed 
in  these  relatively  simple  examples. 

1.3  COMPARISON  OF  CAPABILITIES 

The  relationship  between  men  and  machines  may  be  clarified  by 
listing  some  of  the  functions  in  which  men  surpass  present  day 
machines  and  some  of  the  functions  in  which  present  day  machines 
surpass  men.  Men  excel  in  their  ability  to: 

1.  Sense  or  detect  minimum  amounts  of  visual  or  acoustic 
energy 

2.  Perceive  patterns  of  1 ight,  sound,  or  odors 

3.  Improvise  and  use  flexible  procedures 

4.  Store  large  amounts  of  information  over  long  periods 
and  to  recall  relevant  facts  at  appropriate  times 

5.  Reason  inductively,  and 

6.  Exercise  judgment 

Machines  excel  in  their  ability  to: 

1.  Respond  rapidly  to  control  signals 

2.  Apply  great  force  smoothly  and  precisely 

3.  Perform  repetitive  routine  tasks  rel:  bility 

4.  Store  information  briefly  and  erase  completely 

5.  Reason  deductively,  including  ability  for  computation, 
and 

6.  Handle  highly  complex  operat i ons--many  tasks  at  once 

This  summary  of  the  functional  superiorities  of  both  men  and 
machines  perhaps  indicates  why  there  is  a  growing  belief  among 
engineers  that  we  should  go  to  systems  of  increasing  automati- 
city.  Nevertheless,  man's  superiority  In  adapting  to  changing 
demands  is  one  of  the  fundamental  reasons  why  much  can  be  gained 
from  including  human  elements  in  a  system.  It  appears  likely 
that  for  the  predictable  future  the  human  being  will  continue  to 
be  an  integral  part  of  all  mechanical  or  electronics  systems, 
in  their  operation  and  maintenance.  Therefore  it  is  important 
that  sound  decisions  be  ride  about  his  duties--what  they  should 
be  and  how  they  should  be  performed.  (Figure  14-8) 

We  use  the  term  computer  in  a  general  sense  to  identify  machines 
that  accept  signals  or  data  and  take  specific  action  programmed 
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into  the  machine.  In  this  sense,  automatic  boiler  feed  control 
systems  and  voltage  regulators  are  examples  of  computers.  While 
the  more  complex  computers  are  primarily  electronic  in  nature, 
the  appl icat ion  of  human  capabilities  in  the  design  is  fully  as 
pertinent  in  many  mechanical  systems. 

The  most  important  'aspect  in  which  men  excel  computers  is  in  the 
accessibility  of  the  items  in  storage.  Men  can  apt  at  a  single 
memory  in  many  different  ways;  in  particular,  they  can  recover 
memories  on  the  basis  cf  similarity  alone.  Computers,  by 
contrast,  have  no  such  efficient  c’-oss- indexing .  If  they  did,  it 
would  be  possible  to  write  programs  which  rely  on  the  computer  to 
locate  and  produce  any  item  in  memory  without  specific  instruc¬ 
tion  concerning  where  that  item  is.  At  present,  no  such  procedure 
is  possible. 

A  major  virtue  of  men  is  that  they  have  a  high  tolerance  for 
ambiguity,  vagueness,  and  uncertainty.  Men  are  able  to  detect 
what  other  men  mean  though  the  smog  of  what  they  say,  and  they 
customarily  do  so  and  behave  accordingly.  Such  tolerance  for 
ambiguity  is  based  on  a  life-ion^  history  of  experience  with 
ambiguity  and  on  the  ability  to  argue  by  analogy  from  one's  own 
purposes  to  those  of  other  people.  Neither  of  these  character¬ 
istics  seem  likely  to  be  available  for  computers  in  any  near 
future.  So  long  as  computers  cannot  tolerate  and  exploit 
ambiguity,  they  cannot  be  given  major  executive  responsibilities 
unsupervised;  social  control  is  usually  based  on  vague  mandates 
which  permit  vide  but  not  unlimited  latitude  in  interpretation 
(for  example,  platforms  of  political  parties).  This  means  t.»at 
man-machine  systems  will  necessarily  continue  to  have  men  with 
veto  power  over  computer-generated  decisions,  rather  than  vice 
versa . 

One  reason  why  men  are  good  at  tolerating  and  exploiting  ambigu¬ 
ity  is  that  they  can  effectively  translate  uncertainty  into 
probability--another  task  in  which  mer.  far  excel  computers. 
Consider  the  statement,  "Before  you  go  to  bed  tonight,  you  will 
consume  a  bottle  of  beer."  Presumably  that  statement  is  neither 
impossible  nor  certain.  A  computer  could  probably  go  no  farther; 
a  man  can  attach  a  number  to  the  statement  which  represents  his 
evaluation  of  its  probability  of  being  correct.  Such  numbers 
are,  it  turns  out,  excellent  guides  to  action;  men  can  accurately 
translate  uncertainty  into  probability.  Computers,  on  the  other 
hand,  are  far  superior  to  men  in  taking  probabilities  and  pay¬ 
offs  and  computing  from  them  the  best  course  of  action  in 
accordance  with  rules  set  down  by  man.  These  considerations 
suggest  that  a  mi litary- informat  ion  processing  system  which  must 
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cope  with  relatively  unreliable  data  (such  as  a  sonar  system) 
might  profitably  use  human  operators  as  transducers  for  proba¬ 
bilities.  These  probabilities  could  be  entered  into  a  computer, 
which  would  then  compute  the  optimal  course  of  action  in  the 
light  of  them.  No  such  system  now  exists,  but  it  seems  entirely 
possible  that  they  might  be  one  ten  years  from  now. 

In  a  very  important  sense  men  are  far  more  reliable  than  computers. 
It  has  already  been  pointed  out  that  computers  make  far  fewer 
mistakes  than  men.  But  in  general  the  mistakes  computers  make 
either  remain  unchecked  or  stop  the  computer  completely.  Man,  on 
the  other  hand,  can  detect  his  own  mistakes  and  spontaneously 
work  out  a  plan  to  correct  them  or  remedy  their  effects.  Further¬ 
more,  once  he  has  learned  how  to  perform  a  task  correctly,  man 
does  not  repeat  and  repeat  the  same  error,  as  will  a  computer 
with  a  broken  part.  In  short,  if  a  little  allowance  is  made  for 
the  approximate  nature  of  human  reliability,  man  is  far  more 
reliable  than  any  computer  yet  invented,  or  any  likely  to  be 
invented  in  the  near  future. 

1.4  BASIC  TECHNIQUES  IN  DESIGN 


1.  Allocation  of  functions  among  men  and  computer  should 
consider  the  best  skills  of  each.  It  is  seldom  wise  to  allocate 
to  the  computer  everything  the  designer  knows  how  to  nechanize, 
and  to  parcel  out  among  the  system  operators  whatever  is  left 
over . 


2.  If  possible,  the  computer  should  be  about  80  per  cent  used; 
if  it  has  too  much  unused  time  or  capacity,  either  a  smaller 
computer  should  be  used  instead  or  tasks  tor  which  it  is  less 
than  ideally  suited  (such  as  long-term  memory)  should  be  given  to 
it.  Computers  do  not  profit  from  rest  periods,  other  than  the 
necessary  halts  for  maintenance  and  repair;  men  do.  On  the  other 
hand,  tasks  change,  and  a  little  flexibility  is  therefore  desirable. 

3.  Provision  of  one  sort  or  another  must  be  made  for  system 
function  during  computer  malfunction.  This  often  implies  either 
a  second  computer  or  a  manual  back-up  system.  In  some  cases,  ot 
course,,  no  meaningful  provision  is  possible  or  worthwhile. 

4.  Operator  jobs  should  not  be  homogeneous  in  difficulty.  Some 
jobs  should  require  a  relatively  high  level  of  ability  and  train¬ 
ing;  others  should  not.  This  reduces  the  requirement  for  high-IQ 
operators  and  provides  for  a  career  structure  within  the  system. 

5.  Man-to-man  communications  should  be  carefully  evaluated.  In 
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general,  human  outputs  should  go  into  the  computer*  The  process 
of  man-to-man  communication  is  often  so  clumsy  and  imprecise 
that  the  system  is  sometimes  better  off  if  two  functions  performed 
by  different  men  are  separated  by  a  function  performed  by  the 
computer.  This  principle  is  controversial;  some  experts  insist 
that  a  great  deal  of  informal  man-to-man  communication  is  both 
necessary  and  desirable. 

6.  Man  should  function  as  aids  to  the  computer  in  sensing, 
extrapolating,  and  decision  making.  The  idea  of  using  men  as 
back-up  systems  for  computer  functions  is  very  widely  applied; 
most  manned  space  vehicle  designs  are  designed  that  way.  There  is 
some  question  whether  in  many  applications  it  might  not  be  cheap¬ 
er  and  just  as  effective  to  have  the  man  perform  the  function  in 
the  first  place. 

7.  If  at  all  possible,  the  computer  rather  than  a  man  should 

have  primary  responsibility  for  maintaining  vigilance  and  detecting 
when,  after  a  period  of  inactivity,  some  system  action  is  required. 

8.  A  number  of  specific  tasks  which  must  be  performed  in  most 
information  processing  systems  are  usually  allotted  to  men 
because  they  use  man's  best  skills.  The  functions  of  detection 
and  identification  have  already  been  discussed;  they  exploit 
human  pattern-recognition  ability  and  ability  to  cope  with 
uncertainty.  Another  common  function  is  goal-setting  for  searches. 
Computers  very  often  solve  problems  by  means  of  directed  search 
through  a  very  large  set  of  possible  solutions.  Searches  should 
usually  be  guided  by  hypotheses  conce-ning  the  most  fruitful 

plv.  es  to  search  first. 

9.  Yet  another  important  human  function  in  computerized  systems 
is  censorship.  Men  monitor  the  output  of  computers,  with 
responsibility  to  veto  computer  actions  when  it  seems  appropriate 
to  do  so.  Unfortunately,  as  systems  get  more  complicated  and  their 
tasks  become  irore  demanding,  it  will  oe  more  and  more  difficult 

for  men  to  censor  system  output  effectively.  They  cannot 
assimilate  enough  information  to  be  sure  whether  the  system  is 
right  or  wrong,  except  in  the  case  of  gross  malfunction.  More 
important,  systems  can  seldom  tolerate  the  response  of  doing 
nothing,  and  men  often  cannot  accumulate  the  information  in  time 
to  supply  alternatives  to  the  computer's  recommended  course  of 
action . 

10.  It  will  continue  to  be  true  that  systems  which  include 
computers  exist  to  serve  human  purposes,  so  system  goal-setting 
will  continue  to  be  a  human  function,  the  most  important  human 
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function  in  the  system.  However,  that  function  will  be  performed 
mostly  by  the  designers  of  the  system  and  those  who  write  the 
computer  program;  the  nature  of  system  design  pretty  completely 
determines  the  goals  which  ’ t  can  effectively  further. 

2  *  THE  MAN-MACHINE  INTERFACE 

Having  stated  some  general  principles  applicable  to  the  design  of 
human  tasks,  we  can  turn  our  attention  to  the  ways  in  which  design 
is  actually  made  concrete  within  the  system  development  process. 

It  is  apparent  that  the  human  operator,  whether  functioning  as  ar< 
information  processor,  a  decision  maker,  or  both,  occupies  a 
position  as  a  link  between  two  other  portions  of  the  system.  This 
means  that  he:  a)  responds  to  the  precedina  unit's  output  as  his 
input  and  b)  by  his  action  provides  an  input  to  the  next  unit. 

WTien  provided  by  a  machine,  the  configuration  of  output  events 
that  constitute  input  to  the  human  operator  is  generally  called 
a  display .  The  physical  oljects  which  he  operates  (particularly 
with  his  hands  and  feet'  in  order  to  provide  an  input  to  the 
next  unit  in  the  chain  are  called  controls .  Obviously,  the  way 
the  human  operator  must  function  within  the  system  will  be 
determined  by  the  nature  of  these  displays  and  controls.  Accord¬ 
ingly,  considerations  of  effective  design  for  the  man-machine 
configuration  usually  result  in  decisions  concerning  the  physical 
character ist ics  of  these  aspects  of  equipment. 

2.1  DISPLAY  DESIGN 

The  goal  of  display  design  is  to  provide  the  operator  with 
usable  information  germane  to  his  task  within  the  system.  One 
can  usually  begin  with  the  assumption  that  the  system  has  at  the 
outset  the  basic  means  ot  acquiring  all  the  information  that 
might  conceivably  be  useful.  But  once  the  information  is 
attained,  how  and  in  what  manner  should  it  be  distributed  am on a , 
and  presented  to,  the  human  elements  of  the  system'  The  prob¬ 
lems  can  be  defined  somewhat  more  speci f ical ly .  The  variables 
of  interest  in  the  design  «..»f  displays  have  been  classified  in 
the  following  ways: 

1.  Read  ab  i  1 1 1.  y  ,  l  eg  i  b  i  L  i  t  y  .  Obviously  the  operator  must  be 
able  to  hear  or  see  or  in  some  atlu-i  way  sense  the  signals  being 
provided  for  his  use.  He  must  also  be  able  to  sense  differences 
among  different  signals;  variables  concerning  both  the  display 
proper  and  the  viewing  or  sensing  environment  (for  example, 
illumination)  come  within  this  class. 

2.  -S  e  n  s  o  r  y  m  od  a  1  i  t  y .  The  question  raised  by  this  category 
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concerns  which  sensory  mode  should  be  employed  to  convey  various 
kinds  of  information. 

3.  Multiparametric  or  combined  displays.  Here  are  included 
questions  of  what  and  how  many  different  kinds  of  information  can 
be  incorporated  within  a  single  display  and  how  this  is  to  be 
accomplished  most  effectively. 

4.  Display  coding.  This  category  implies  questions  as  to  the 
language  form  or  the  kind  of  symbols  to  be  employed  in  presenting 
information. 

5.  Filtering .  Questions  of  this  class  concern  ways  of  pre¬ 
selecting  information  inputs  so  as  to  simplify  the  interpretation 
task . 

6.  Clutter  and  noise.  Included  in  this  category  are  problems 
pertaining  to  the  elimination  of  false  or  masking  signals  in  the 
display . 

2.2  DESIGN  OF  CONTROLS 


The  ruling  concepts  in  control  design  should  be  order,  coherence, 
and  organization.  Rather  than  flexibility  or  changeability,  one 
wants  control  devices  to  have  the  properties  of  being  orderly 
and  consistent  in  their  operation  and  action  consequences.  From 
the  standpoint  of  the  human  operator,  perhaps  predictability  may 
be  thought  of  as  the  most  desirable  characteristic. 

The  layout  of  control  panels  and  consoles  is  a  good  place  to 
begin  consideration  of  control  design  problems.  The  basic 
technique  for  the  designer  consists  in  analyzing  the  task.  The 
task  analysis  provides  a  map  of  what  the  operator  is  supposed  to 
do  in  carryout  out  his  job  Traditi onal iy  the  analysis  is  a 
description  of  isolated  actions  in  sequence.  For  relatively 
simple  tasks  (for  example,  mechanical  assembly)  the  classic 
"therblig"  of  industrial  engineering  is  appropriate.  With 
increasing  operational  complexity,  as  well  as  the  necessity  to 
develop  equipment  for  tasks  which  are  almost  entirely  novel, 
newer  techniques  are  needed.  By  whatever  means  obtained, 
however,  an  analytic  map  of  the  task  is  essential. 

Just  as  the  system  as  a  whole  may  be  functionally  organized 
according  to  operations,  the  control  layout  can  be  so  organized. 
Functional  grouping,  ease  of  access,  differentiability  of  sub¬ 
operations,  and  the  like  then  come  into  play  as  criteria  for 
console  design.  Unfortunately,  the  criteria  are  not  always  (in 
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the  controls  to  which  the  individual  responds,  and  the  mechanisms 
which  provide  the  transformations  for  these  components  of  the 
system  are  of  considerable  variety.  Accordingly,  we  need  to 
recognize  at  once  that  the  generalized  picture,  while  it  indicates 
man's  position  as  a  system  component,  does  not  provide  the  means 
for  a  detailed  analysis  of  the  variety  of  human  functions.  It 
would  be  a  mistake  to  think  that  because  man  typically  "occupies 
a  space"  between  machine  displays  and  controls,  his  functioning 
can  be  related  in  a  constant  set  of  ways  to  such  inputs  and  out¬ 
puts.  The  fact  is,  neither  the  input  nor  the  output  by  them¬ 
selves  will  tell  us  the  nature  of  man's  functioning.  For  there 
are  different  kinds  of  transformations  which  may  be  performed 
(by  the  human  nervous  system)  in  turning  inputs  into  outputs. 

1.2  FUNCTIONS  PERFORMED 

Suppose  that  a  system  has  available  an  oscilloscope  as.  a  basic 
unit  for  display.  On  the  face  of  this  scope  appears  a  60-cycle 
wave  form  which  can  be  adjusted  in  amplitude  to  a  particular 
size,  the  required  size  being  given  by  two  fine  horizontal  lines 
on  a  transparent  overlay  placed  against  the  tube  face.  Let  us 
assume  that  the  system  requires  the  amplitude  of  the  wave  to  be 
determined  within  very  close  tolerances  before  further  operation 
of  the  system  can  take  place.  Following  this,  what  happens  is 
that  an  external  signal  distorts  the  form  of  the  wave,  and  it  is 
the  human  operator's  task  to  "report"  the  nature  of  these 
distortions  by  pushing  one  of  five  buttons.  If  there  is  an 
amplitude  distortion  (vertical  displacement  from  the  overlay 
markers),  he  pushes  button  1?  if  there  is  a  frequency  distortion 
(horizontal  displacement  from  other  markers),  he  pushes  button  2; 
if  there  are  additional  frequencies  present  (irregular  wave 
pattern),  he  pushes  button  3;  and  so  on. 

Now,  saving  certain  details  for  later  consideration,  let  us 
consider  the  difference  between  what  the  operator  does  in  "getting 
the  equipment  ready  to  operate"  and  what  he  does  in  "operating." 
Actually,  he  is  utilizing  two  different  functions  in  the  two 
cases,  even  though  the  display  may  be  the  same  in  both. 

In  placing  the  equipment  into  proper  operating  condition,  the 
human  operator  is  making  use  of  the  function  of  sensing.  That 
is  to* say,  he  is  using  his  visual  receptors,  nervous  system,  and 
effectors  simply  to  "report"  (to  a  machine,  typically)  the 
presence  or  absence  of  a  difference  in  physical  energy.  In  this 
case,  the  physical  difference  being  reported  is  the  coincidence 
of  two  points  (or  small  areas)  each  of  which  lies  along  a  narrow 
band  of  light  (a  "line")  which  makes  an  abrupt  gradient  of 
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6.  Display  compat ibil ity .  In  the  total  configuration  of  displays 
and  controls  making  up  the  operator  station,  display  control 
arrangement  should  be  correlative;  that  is,  controls  governing 
the  process  being  displayed  should  be  in  proximity  to  their 
related  display.  Display  format  and  con*  ent  should  be  dimension¬ 
ally  similar  to  control  location  and  direction  of  action. 

As  our  previous  discussion  has  suggested,  it  is  often  not 
possible  to  follow  all  these  rules  with  equal  vigor.  The  d-  gn 
of  controls  and  control  layouts  is,  of  course,  a  matter  of 
practical  compromise,  as  is  true  of  other  aspects  of  eauipment 
design.  Nevertheless,  this  set  of  principles  represents  the 
factors  that  are  based  on  empiri  il  findings  of  studies  of  human 
functioning,  which  can  successfully  be  brought  to  bear  on  design 
decision  having  the  aim  of  optimal  system  effectiveness. 

2.3  REDUCING  HUMAN  ERROR 

On  the  whole,  our  discussion  of  design  problems  has  been  carried 
out  in  the  context  of  human  functioning,  and  particularly  in 
consideration  of  the  ways  of  eliminating  the  kinds  of  function¬ 
ing  that  result  in  error.  The  analysis  of  poss ibi 1 ■ t les  of 
operator  error  leads  to  the  conclusion  that  its  causes  may  often 
be  identified  as  deficiencies  in  equipment  design,  whether  of 
displays,  controls,  or  the  expected  Interactions  between  these 
two  types  of  elements.  Trie  information  obtained  from  such 
analyses  is  exhibited  in  summary  from  i n  Figure  14-16 .  It  will  be 
noted  that  many  of  these  difficulties  provide  the  possibility  of 
correction  by  means  of  equipment  design  (for  example,  unclear 
code  form! ,  whereas  others  would  appear  to  be  avoidable  by  the 
provision  of  external  instruct  ions ,  possibly  by  means  of  lob 
aids  (for  example,  inappropriate  filtering  sot}.  In  still 
other  instances,  it  appears  that  corrective  action  would  take  the 
form  of  training  to  be  undertaken  after  the  equipment  configura¬ 
tion  lias  been  determined  (for  example,  action-control  relation¬ 
ships  not  understood  by  the  operator). 


It  seems  evident  from  the  figure  that  there  are  a  number  of 
ways  of  preventing,  minimizing,  or  reducing  the  deleterious 
effects  of  peratcr  mistakes,  when  one  recognizes  that  such 
errors  may  ue  understood  as  matters  of  inadequate  functioning 
of  human  in  format  ion- process ing  and  decision-making  activities. 
In  many  cas*u  ,  the  avoidance  of  faulty  human  functioning  can  be 
specifically  related  to  the  design  of  equipment  ^isp1 aye  and 
controls,  and  particularly  to  the  extent  to  which  they  define 
sensible  human  tasks.  If  one  look.-,  beyond  these  principles 
(paragraph  2,2),  he  can  foresee  the  possibility  c  f  a  systematic 


TYPICAL  HUMAN  ERRORS 


Error  of  commission  Correct  tool  or  control  not  a\ ;n inhlc 

•Action-control  relationship  not  understood  i>y  operator 


theory  of  human  performance,  for  which  currently  acceptable 
categories  cf  human  functioning  provide  only  the  bare  framework. 

3 .  EVALUATION  OF  MAN  IN  THE  SYSTEM 

It  has  been  pointed  out  earlier  that  engineering  specialists 
utilize  tests  as  a  practical  way  of  reducing  uncertainty  and 
providing  feedback  about  comp^nen^,  subsystem,  or  system 
performance.  It  is  a  means  whereby  information  is  obtained  about 
the  validity  of  design  decisions.  It  is  also  a  way  to  advance 
the  state  of  the  art  by  a  trie-  and  error  approach  when  a  more 
rigorous  scientific  approach  is  not  feasible,  as  in  the  time  and 
cost  environment  surrounding  development  programs.  The  wide  use 
of  engineering  test  programs  has  also  led  to  the  development  of 
standard  tests,  standard  test  procedures,  and  improved  test 
instrumentations  which  permit  mere  elegant,  more  pertinent,  and 
more  economical  testing.  E'or  example,  over  300  flight  tests 
wtre  run  on  the  V2  to  achieve  desirable  performance  character¬ 
istics.  The  Sergeant  missile,  which  was  later  developed  by  many 
of  the  same  individuals  who  developed  the  V2 ,  performea  adequate¬ 
ly  after  approximately  50  flight  tests. 

Testing  has  also  uncovered  facts  that  were  not  or  could  not  be 
foreseen  by  pretest  logical  analyses.  For  example,  on  one 
program  there  were  a  number  of  tests  which  ended  with  the  missiles 
breaking  up  in  mid-air.  It  was  only  after  sufficient  testing 
that  the  reason  for  disintegration  was  identified  as  analytically 
unexpected  torsional  bending  stress.  As  a  result,  current  missile 
programs  collect  data  on  this  parameter  as  a  mattar  cf  course. 

Engineering  test  activities,  briefly,  include:  (a)  making 
predictions  about  the  performance  of  the  system  (or  of  some 
subsystem  or  component) ,  these  predictions  being  usually  based 
upon  interpolation  or  extrapolation  of  established  data;  (b) 
designing  tests  to  confirm  predictions;  (c)  instrumenting  the 
test  area  and  vehicle  in  order  to  acquire  pertinent  data  for  the 
evaluation  of  the  predictions;  (d)  evaluating  and  comparing,, 
predicted  performance  with  actual  performance;  and  (e)  employing 
the  results  of  test  as  the  basis  for  analysis  of  the  discrepan¬ 
cies  between  predictions  and  empirical  results  and  to  modify  the 
model  from  which  the  predictions  were  made,  or  the  system,  or 
both. 

It  should  be  pointed  out  that  there  is  a  difference  in  intent  and 
in  criteria  between  the  type  of  "testing"  that  is  involved  in  an 
engineering  development  program  and  the  type  of  "testing"  that 
is  involved  in  traditional  research  experimentation.  The 
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engineering  test  program  has  as  its  primary  concern  the  achieve¬ 
ment  of  an  "adequate"  system  where  "adequate"  is  defined  as  a 
system  that  meets  design  objectives.  In  this  context,  one  is 
not  primarily  concerned  with  determining  "good"  design  or 
"optimum"  design,  or  with  collecting  basic  data,  but  only  with 
the  question,  "Does  the  system  meet  required  and  predicted  per¬ 
formance  criteria?"  It  is  assumed  that  the  designer  and  design 
management  are  concerned  with  providing  the  best  design  possible, 
and  that  they  make  use  Oi  available  experimental  data.  Thus, 
tests  must  answer  th  question,  "Does  the  design,  which  is 
assumed  to  be  the  best  that  the  designer  could  produce  at  the 
time  the  design  decision  was  made,  meet  the  design  requirements?"' 
An  example  in  human  engineering  terms  would  be  a  tracking  opera¬ 
tion  in  which  the  operator  is  required  to  track  two  objects  within 
certain  time  and  accuracy  limits.  The  human,  engineer  would 
provide  the  "best"  information  he  had  at  his  disposal  at  the 
time  a  decision  to  include  such  an  operation  was  required.  The 
purpose  of  human  engineering  test,  then,  would  be  to  determine 
whether  operators  dc  indeed  perform  within  the  specified  limits 
and  not  whether  the  design  that  was  recommended  turns  out  vo  be 
the  optimum  design  for  the  circumstances  under  which  it  is 
finally  used. 

3.1  HUMAN  FACTORS  ENGINEERING  TDDTING 

Performance  testing  and  malfunction  reporting  attempt  to  provide 
workable  ways  of  arriving  at  quantifying  the  effects  of  the  man 
on  the  system.  Most  malfunction  data  collection  systems  are 
already  making  some  attempt  to  obtain  data  on  human- in  it iated 
malfunctions.  Consequently,  the  following  approach  is  based  on 
modification  or  extension  of  existing  malfunction  data  collec¬ 
tion  systems. 

3 .1.1  Identifying  and  Describing  Human  Performance:  The 
achievement  of  adequate  over-all  system  performance  within  the 
constraints  of  given  dollars  and  time  is  the  primary  task  of  a 
system  development  program.  However,  the  size  and  complexity 
of  modern  weapon  systems  make  it  necessary  to  subdivide  the 
system  and  its  developmental  task  into  manageable  parts  upon 
which  can  be  brought  to  h^ar  the  varied  capabilities  of  many 
individuals  and  groups.  The  parts  or  subsystem  entities  that 
are  most  familiar  are  the  equipment  "packages"  in  the  form  of 
subsystems,  components,  or  their  smallest  "bits  and  r ' eces . " 

These  provide  the  equipment  designer  with  discrete  entities  that 
can  be  separately  analyzed  and  designed  within  the  context  of 
the  system  as  a  whole,  and  for  which  performance  predictions  can 
be  made  and  tested.  Alone  or  in  various  assemblages,  these 
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hardware  entities  are  used  as  the  primary  vehicles  for  test 
design,  test  data  collection  and  analysis,  and  any  subsequent 
system  modi fications . 

From  the  viewpoint  that  man  participates  in  weapon  system 
functions  through  the  operations  he  performs,  it  appears 
appropriate  to  use  the  operation  as  the  entity  that  is  the  human 
factors  engineering  equivalent  of  the  hardware  designer’s 
"black  box.  ' 


3.1.2  Performance  Testing:  In  order  to  develop  a  human 
factors  engineering  test  prc/ram,  a  number  of  problems  have  to 
be  considered.  The  approach  to  human  factors  engineering 
performance  testing  proposed  here  considers  tve  following: 

1.  Identification  and  selection  of  critical  human 
operations . 

2.  Specification  of  pertinent  parameters  of  these 

oper at ions . 

3-  Prediction  of  the  values  of  these  parameters. 

4.  Confirmation  through  test  of  predicted  parameter 
values . 

5.  Adequacy  of  test  data. 

6.  Test  implementation. 

3.1.3  Identification  of  Critical  Human  Operations  :  Critical 
operations  as  a  subcategory  of  all  human  operations  can  be 
defined  as  those  which,  if  not  performed  in  accordance  with 
estimated  design  values,  will  most  likely  have  large  effects 
on  a  system's  performance  or  cost. 

3.1.4  Selection  of  Critical  Operations:  For  systematic 
consideration,  the  operations  that  humans  perform  in  a  system  can 
be  organized  and  presented  in  the  form  of  block  diagrams.  Each 
operation  must  be  considered  and  a  decision  made  as  to  whether 
that  operation  is  to  be  included  for  evaluation  in  the  test 
program.  A  typical  priority  list  would  consider  such  factors  as: 

(a)  Past  Performance:  If  the  man  or  man-machine  operation  is 

in  all  essential  aspects  similar  to  man  or  man-machine  operations 
of  previously  evaluated  weapon  systems,  this  previous  experience 
would  be  important  in  determining  the  necessity  for  including  a 
test  of  the  operation  in  the  test  program.  Past  malfunction 
experience  can  provide  a  useful  guide  in  this  regard. 

(b)  Value  Loss:  The  estimated  amount  of  time,  accuracy,  or 
cost  penalty  that  may  result  from  an  operation  that  is  performed 
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imperfectly  is  another  factor  that  must  be  considered  in  deter - 
mininu  the  test  priority  of  that  operation.  The  priority  for 
inclusion  in  the  test  program  is  highest  for  these  operations 
which  have  the  greatest  estimated  time,  accuracy,  or  cost  penally 

associatea  'with  them. 

(c)  Test  and  Evaluation  Cost;  Since  the  tests  of  man  or  man- 
machine  operations  are  tests  of  components  and  subsystems  of  tne 
over-all  weapon  system,  tests  of  these  components  and  subsystems 
can  often  be  included  in  over-all  system  or  subsystem  tests. 

3.1.5  Specification  of  Parameters;  Once  a  selection  has  been 

,|.l  .  —  — - —  .  — —  . 

made  of  the  critical  human  operations  of  the  weapon  system  that 
are  to  be  subjected  to  performance  testing,  these  operations 
must  be  described  in  an  appropriate  way.  This  ir.e«ns  the  opera¬ 
tions  parameters  must  be  selected  and  specified  in  a  way  which 
adequately  describes  the  operations  and  specifies  the  criteria 
co  be  applied  so  that  the  operations  are  measurable  in  test  and 
so  that  they  are  subject  to  modification  for  system  improvement. 

3.1.6  Prediction  of  Parametric  Values;  After  the  important 
parameters  associated  with  the  critical  operations  to  be  tested 
have  been  -pecified,  the  values  that  these  parameters  may  be 
expected  to  assume  are  predicted.  There  are  few  formulas  avail¬ 
able  which  will  allow  the  calculation  of  exactly  how  fast  or 
how  accurately  a  man  will  perform  a  certain  operation  in  a  given 
situation . 

I f  no  formal  or  semi  formal  model  of  an  operation  is  established, 
then  a  tester  would  be  forced  to  take  data  over  the  entire  range 
of  each  parameter  and  for  combinations  of  parameters.  Good  test 
design  is  the  art  of  predicting  (hypothesizing)  in  such  a  manner 
that  with  a  minimum  of  effort  a  maximum  of  critical  information 
can  be  derived. 

No  matter  what  model  or  method  is  used  to  aid  in  predicting 
operator  performance,  the  model  or  method  must  account  for  the 
value  that  a  particular  parameter  may  take  under  certain  con¬ 
ditions. 

3 . 2  CON PI RMATIQN  OF  PREDICTIONS  THROUGH  TESTING 

Models  that  are  used  to  predict,  operator  performance  can  be 
confirmed  only  through  tests.  The  need  for  such  confirmation 
is  inversely  proportional  to  the  confidence  which  can  be  placed 
in  the  predictive  model.  Thus,  in  evaluating  a  system,  it  would 
be  regarded  as  unnecessary  to  prove  through  tesl  che  validity  of 
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Ohm’s  or  Boyle's  Law;  these  relationships  are  well  enough  estab¬ 
lished  to  be  accepted  without  any  further  confirmation.  If,  on 
the  other  hand,  the  predictive  model  is  no  more  than  a  "guessti¬ 
mate,"  or  a  statistical  model  with  wide  variances,  then  additional 
testing  to  confirm  these  predictive  models  will  be  necessary  to 
give  the  user  confidence  in  the  system  design. 

Tests  leading  to  the  confirmation  of  predictions  can  be  defined 
as  a  series  of  operations  employed  to  determine  the  correspon¬ 
dence  between  a  model  which  describes  an  existing  or  potential 
state  of  affairs  and  the  actual  state  of  affairs.  A  model  (or 
theory)  implies  a  hypothesis  or  a  set  of  hypotheses. 

3.2.1  Adequacy  of  Test  Data;  Confirmation  of  predictions  is 
dependent  upon  the  adequacy  of  test  data.  It  will  be  useful 
therefore  to  discuss  briefly  several  factors  that  are  involved 
in  test  adequacy. 

The  amount  and  the  validity  of  information  obtained  from  test 
often  rests  on  the  sophistication  that  goes  into  designing  the 
test.  Frequently,  it  is  sufficient  to  run  tests  under  normal 
operating  conditions.  Under  other  circumstances,  it  may  be  more 
useful  to  "vest  to  breakdov..";  that  is,  to  test  under  abnormal 
conditions  where,  for  example,  deliberate  overloads  and  mal¬ 
functions  are  introduced.  This  may  be  necessary  to  determine 
the  range  within  which  certain  operations  can  be  performed  with 
a  specified  speed  or  accuracy  by  human  or  other  system  components. 

Once  a  major  source  of  variance  has  been  detected,  it  is  likely 
that  something  positive  can  be  done  to  reduce  it**  magnitude. 
However,  certainty  about  the  original  source  of  a  variance 
measure  can  be  approximated  only  by  more  investigation  where, 
for  example,  specific  significant  interactions  are  rigorously 
examined  and  investigated. 

Test  Implementation:  In  the  absence  of  a  large  body  of 
experience  with  human  factors  performance  test  from  which  to 
draw  examples,  many  questions  arise,  For  example,  who  should  do 
human  factors  testing, the  human  factor  specialist,  a  human  factor 
test  technician,  or  regular  test  personnel?  It  seems  obvious 
that  the  peculiarities  of  obtaining  reliable  data  on  human  per¬ 
formance  requires  the  attention  of  someone  with  training  in 
experimental  methods  in  the  behaviorial  sciences,  particularly 
in  psychology,  yet  the  actual  collection  of  data,  at  a  test  site 
may  be  placed  in  the  hands  of  someone  with  limited  training  in 
obtaining  behavioral  data. 


14-22 


3 ♦ 3  EXTENSION  OF  CURRENT  FAILURE  REPORTING  P ROCEDURE3 

For  malfunction  data  to  be  useful  in  the  improvement  of  system 
performance,  they  must  be  sufficient  for  identifying  the  human 
factors  involved  in  the  reported  failure  events.  By  identifying 
these  factors,  the  data  make  it  possible  to  recreate  the  dynamics 
of  the  situation  in  which  a  failure  has  occurred  and  to  determine, 
analytically  or  physically,  what  may  have  caused  it  and  v?hat 
steps  to  take  to  prevent  its  recurrence.  To  do  this,  the  data 
concei  ning  the  individual  failure  must: 

1.  Identify  the  failed  item. 

2.  Described  the  symptoms  by  which  the  failure  was  identified. 

3.  Provide  a  means  for  describing  the  dynamic  interactions  of 

personnel  with  the  failed  item,  with  other  parts  of  the 
system  and  with  the  system's  environment. 

4.  Provide  information  concerning  the  past  experience  of  the 

individual  failed  item  that  might  be  pertinent  to  the 
failure  event. 

5.  Record  the  skill  level,  or  ratinq  of  the  technician 

diagnosing  the  failure  and  making  the  restoration. 

3.4  HUMAN  RELIABILITY 

The  Personnel  Subsystem  concept  emphasizes  the  development  of 
human  performance.  Human  reliability  can  be  compared  to  hardware 
reliability  in  the  system  reliability  program.  Human  performance, 
like  hardware  performance,  must  satisfy  the  performance  require¬ 
ments  of  the  system. 

In  aerospace  systems  we  are  approaching  an  interchangeability 
for  man  similar  to  that  which  exists  for  hardware.  From  the 
performance  aspect  man  has  become  a  relatively  standardized  com¬ 
ponent  in  the  system.  He  must  function  within  a  given  range  of 
tolerance  to  satisfy  systei..  requirements.  Consequently ,  every 
individual  or  group  of  individuals  in  the  system  must  be  capable 
of  providing  the  required  per formance .  Personnel  variability 
and  interchangeability  directly  affect  the  operational  capabil¬ 
ity  of  the  system.  Variability  of  the  human  component  must  be 
either  minimized  or  have  onl*  negligible  effect  on  system  per¬ 
formance  if  the  potential  system  capability  is  to  be  fully 
realized . 

The  previous  tendencies  for  system  development  to  be  primarily 
concerned  with  on Ly  hardware  development  or  with  the  assumption 
that  hardware  test  data  adequately  evaluate  the  Personnel  Sub¬ 
system  have  made  it  extremely  difficult  to  empirically  identify 
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human  errors  and  the  effects  of  human  variability  on  the  system. 
However,  some  information  on  the  amount  of  human  error  does 
exist.  For  example,  the  Shapero  (3)  Study  indicated  a  range 
from  20  to  53  percent  for  human- initiated  malfunctions.  An  ana¬ 
lysis  of  600  recent  rocket  engine  Failure  and  Consump- ion  Reports 
showed  that  3  5%  of  the  failure  reports  indicated  equioruent 
damage  or  malfunction  directly  attributable  to  hum^n  interaction 
with  the  equipment  during  maintenance,  checkout,  and  transport. 

Up  to  40%  of  all  missile  holds,  postponements,  and  failures  are 
caused  by  h  man  error.  The  published  studies  involved  a  further 
analysis  or  r-  an,  ysis  of  data  previously  collected  rather  than 
the  more  positive  effort  to  actually  collect  and  classify 
Personnel  Subsystem  data  as  an  integral  part  of  the  failure- 
data  reporting  system.  If  such  an  analysis  can  be  performed  by 
a  group  of  individuals  in  retrospect,  it  should  be  equally  pos¬ 
sible  to  perform  the  analysis  at  the  time  the  data  is  originally 
collected  --  providing  methods  and  techniques  are  adequate  for 
’dent. if  irj  human  initiated  failure  on  the  failure-data  reports. 

Confining  human  reliability  to  the  effects  of  human  performance 
structuies  tue  concept  but  docs  not  solve  the  reliability  pro¬ 
blem.  Human  reliability  is  not  easily  predicted  nor  controlled. 
Huma"  failure  is  not  identical  to  equipment  failure.  Peop1e  are 
not  fixed  components.  Alter  human  performance  has  once  been 
established  human  failures  tend  to  be  intermittent.  Usually  the 
indivir  tal  who  fails  to  perform  a  specific  ta^k  at  a  specific 
time  wiri  perform  the  task  correctly  the  next  time.  Intermittent 
human  failure  plagues  the  operational  situation.  human  perform¬ 
ance  is  affected  1  /  many  complex  factors  such  as  motivation, 
stress,  and  tatigue.  These  complex  factors  are  difficult  (but 
not  impossible)  to  predict  and  control. 


4.  CONCLUS ION 

Although  nch  generally  recognized,  the  birth  of  the  new  era, 
symbolized  by  the  term  man-machine  systems,  occurred  when  the 
tcchnolaists  produced  the  automatic  tracking  and  fire  control  or 
un  laying  radars.  This  was  quickly  followed  by  the  second 
technological  innovation,  high  speed  electronics  computers. 

With  these  machines  we  had  devices  which  could  replicate  the 
logic  process  which  heretofore  had  been  the  exclusive  domain  of 
man . 

With  the  invasion  by  the  machine  into  the  logic  process,  the 
relativ  roles  of  man  and  machine  have  undergone  a  subtle  but 
nonetheless  fundamental  change.  No  longer  can  we  regard  man  as 
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an  entity  apart  from  the  system  --  an  entity  wno  operates,  main¬ 
tains  or  controls  the  machine.  Rather  he  is  explicitly  a  part  of 
the  system  contributing  those  capabilities  which  are  uniquely 
his.  Thus  ii.  *•'  nory  at  least  we  now  have  man-machine  systems 
with  the  man  as  .igned  those  tasks  which  he  can  do  most  effective¬ 
ly  and  efficiently  and  the  machine  assigned  there  tasks  which  it 
can  do  most  effectively  and  efficiently. 

Man  is  subject  to  the  machine  even  as  the  machine  is  to  him. 
through  the  interactions  which  take  place  in  today's  complex 
systems.  Certainly  man's  judgment  must  prevail  and  in  a  sense 
can  be  considered  to  control  since  the  machine  does  not  possess 
intellect.  However,  we  must  not  lose  sight  of  the  fact  a at 
even  this  "man-only"  attribute  can  be  and  is  influenced  to  a 
remarkable  extent  today  by  the  met nod  of  processing  and  manner 
of  display  of  the  pr  ^cessed  data  by  the  machine. 

Within  the  context  of  our  philosophical  concept  Human  Factors 
Engineering  is  a  very  broad  ana  of  concern  encompassing  diver  r 
disciplines  in  the  behavior  .1  cionces ,  physiology,  anthropo¬ 
metries  and  psychometrics.  I  an  applied  Engineering  sense  it 
includes  the  area  which  we  re.er  to  as  Personnel  Management  nd 
Training.  Actually,  one  can  co’  ceive  of  the  Personnel  and 
Training  people  as  being  the  producers  of  the  man-modules  for 
our  systems.  It  is  to  them  that  our  systems  engineers  look  fo. 
the  man  in  the  systems.  It  is  to  them  also  that  the  systems 
engineers  look  for  the  descriptive  specifications  of  ‘■he  man 
available  for  incorporation  into  the  system. 

Herein  lies  the  problem.  While  there  is  a  positive  e.  ort  to 
provide  quality  control  in  processing  the  product  and  n  selec¬ 
tion  of  the  raw  material  input,  the  random  nature  of  the  origins 
of  the  raw  material  poses  real  difficulties.  As  a  result,  des¬ 
criptive  specifications  are  given  in  very  broad  parameters. 

This  situation  is  aggravated  by  our  lack  of  real  understanding 
as  to  how  and  why  this  raw  material,  man,  functions.  Neither  do 
we  have  the  attendant  measuring  systems  for  this  functioning. 

We  point  with  pride  to  the  fine  tolerances  to  which  we  can  pro¬ 
duce  machine-elements.  We  measure  them  with  micron  exactness. 
Then  we  ask  the  system  designer  to  combine  them  with  man-elements 
which  we  describe  as  an  average  man  with  an  bth  grade  mentality. 
What  precision!  What  an  exquisitely  defined,  measurement  scale! 

--  urid  management  says,  "Give  us  systems  effectiveness." 

In  order  to  reach  the  design  goal  we  must  first  learn  far  more 
than  we  now  know  about  how  and  why  a  man  functions.  We  must 
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learn  how  to  measure  the  parameters  which  describe  these-  Funct  i  eg. 
We  must  acquire  the  capability  to  describe  exact ly  what  combina¬ 
tions  of  man- functions  are  (or  potentially  are)  in  ur  avail abb' 
inventory  together  with  the  distributions  of  tnose  functions. 
Until  we  are  able  to  provide  adequate  man  parameters  to  our 
systems  designers,  the  probabilities  of  true  systems  effective¬ 
ness  will  continue  to  be  quite  low  and  high  systems  effectiveness 
will  be  more  accidental  than  calculated.  T  ow  systems  effective¬ 
ness,  I  submit,  is  the  situation  today.  This  is  muni  f opted  in  a 
myriad  of  reports  (2). 


These  reports  use  such  terms  as  "too  complicated  machines1, 
"inadequate  training",  " above  the  heads  of  our  people1,  "cv. 
maintained",  etc.  Thei  ,  if  you  will,  are  sympt  -  ms  .  Th.es  • 
demonstrate  our  inability  to  fit  the  available  man-modules 
the  system.  In  almost  every  case,  we  are  able  tc  provide  a. 
bination  of  man-modules  and  machine-modules  that  iocs  funct 
effectively.  More  often  than  not  .ve  ascribe  the  di  ? feror.v 


c  cm  - 
ion 
be¬ 


tween  the  successes  and  failures  to  such  tilings  as  leaders!*.  ip, 
luck  or,  in  come  cases,  a  unique  set  of  circumstances .  In 
case,  one  effective  system  case  is  evidence  that  the  system 
work  effectively. 


•roblem  to  e 


Quantising  the  man  function  then  becomes  the  core 
systems  effectiveness  effort.  What  do  we  do  about 


nd  t valuation  of  the  man-  funct ion. 

If  we  are  to  resolve  the  probLem ,  we  mu 
study  and  analysis  of  the  man  parameter 
mar*  that  which  we  current  ly  have  undei 
g.-.o  in  our  understanding  ana  mea.-uiremon 
We  must  initiate  and  support  ef fort  s  in 
wil:  lead  to  an  understanding  an  i  me  nu; 
that  we  possess  for  the  gear,  the  elect 
acid  - 


an  attempt  to  resolve  the  problem,  the  Bureau  of  Ships  st;  aportoc 
by  the  Office  of  Naval  Material  has  initiated,  a  protect.  c>.lled 
TRIM.  TRIM  is  an  acronym  for  Training  Requirements  Inform,  t. ion 
Management.  TRIM  is  a  systematic  approach  to  the  c>.  >d  i  t  ioat. .  on  , 
recording  end  collection  of  training  requirements  data  ind 
personnel  resource  data  in  terms  of  training.  Perhaps  the  most 
significant  aspect  of  TRIM  is  that  its  design  concept  takes  mtc 
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account  the  gross  nature  of  existing  measures  of  man  parameters . 

As  a  result  the  matrices  in  the  system  have  been  designed  to 
provide  for  ultimately  m  >r  ret  mod  measures  without  necessitnt  in  i 
a  new  data  s  y s  t  cm . 

A  second  Navy  project,  which  I'd  like  to  cite, is  the  effort  under 
the  sponsorship  of  the  Chief  of  Naval  Personnel  referred  to  as 
the  New  Developments  Human  Factors  Program.  This  is  a  rather 
broad-gauged  effort  to  define  the  problem  and  provide  solutions 
in  the  personnel  management  and  training,  or  i f  you  will,  pro¬ 
duction  processes  for  our  man-modules. 

However,  the  vast  bulk  of  our  military  systems  must  use  the  so- 
called  average  man.  Further,  highly  specialized  and  very 
expensive  artificial  environments  are  s imply  not  economically 
feasible  for  them. 

Therefore,  we  must  learn  more  about  how  and  why  this  average 
man  performs.  We  must  learn  how  to  measure  and  predict  this 
performance.  These  measures  and  these  predict  ions  may  then  be 
used  by  the  system  designer  as  the  descriptive  parameters  of  the 
man  in  the  system.  The.,  and  only  then  can  we  hope  to  achieve 
overall  systems  effectiveness  in  our  military  systems . 
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Chapter  15 
DESIGN  REVIEW 


1.  PHASES  OF  DESIGN  REV  [EV 

1.1  Conceptual  Phase  of  Design 

1.2  Preliminary  Design 

1.3  Formal  Design  Review 

1.4  Final  Des 1  an 

1.5  Alteration  or  Corrections 


15-  3 

if-  -> 

1  ">  -  > 

15-  4 
1  5-  4 
15-  7 
1  5-1  . 


2.  CHECKLISTS 

.1  Data  Package  Checklist 

.2  General  Design  Checklist 

2.3  Structural  Fatigue  Checklist 

2.4  Human  Factors  Checklist 

2.5  Development  of  Checklists 

3.  COVERAGE  OF  DESIGN  REVIEW 

3.1  Maintainability 

3.2  Parts  Control 

3.3  Manutactur  itvt  Process  Engineers 
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3.5  Duality  Control  Engineers 

4.  EFFECTIVENESS 
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DESIGN  REVIEW 

The  objective  of  design  review  is  mature  design  the  first  time. 

We  are  trying  to  accelerate  the  brainchild's  growth  and  learning 
rate,  and  to  know  when  and  whether  it  can  reliably  stand  on  its 
own  feet . 

We  want  to  help  the  design  engineer  to  "think  of  everything." 

While  this  may  be  humanly  impossible  when  he  is  yourg.  working 
alone  or  without  discipline,  we  can  come  very  close  to  "every¬ 
thing"  with  systematic  assistance  and  guidance. 

We  also  want  to  make  sure  that  the  very  best  available  brains 
are  applied  to  the  design.  By  brains  we  mean  both  competence 
and  experience.  Most  engineers  realize  that  they  do  not  know 
everything  about  their  area  of  design.  The  very  fact  that  they 
are  designing  implies  the  need  to  create  new  and  untried  things. 
But  young  engineers  often  cannot  appreciate  what  they  do  not  yet 
know.  So  we  must  find  ways  to  make  Tthe  knowledge  of  experienced 
engineers  not  only  handily  available  to  the  design  engineer,  but 
also  invariably  and  systematically  used. 

Also  it  is  desirable  to  subject  the  design  to  the  different  view- 
points  arising  from  experience  in  different  fields  or  disciplines. 
An  expert  in  one  field  can  repeatedly  miss  a  flaw  that  is  obvious 
to  the  expert  in  another. 

Achievement  of  this  objective  will  of  course  protect  the  Bureau 
and  fleet  from  the  cost  and  delay  of  preventable  errors,  protect 
the  contractor  from  much  Bureau  unhappiness,  and  protect  the 
contractors  community  from  degenerate  employment  resulting  from 
consequent  loss  of  business. 

The  manner  in  which  design  reviews  are  conducted  differs  mainly 
in  degree  of  formalization.  A  conversation  between  a  designer 
and  a  friend  in  another  code  may  very  well  accomplish  the  main 
purpose  of  a  design  review.  On  the  other  hand  requiring  that  an 
engineer  complete  detailed  design  review  questionnaires  solely 
to  justify  his  design  decisions  does  not  usually  accomplish  any 
purpose.  Formality  should  probably  be  confined  to  formal  re¬ 
quirements  of  scheduled  events  (review,  report,  follow-up)  and 
content  of  the  reviews  but  not  extended  to  fc>  s.*at  for  those 
events . 

It  must  be  remembered  that  a  design  review  deals  with  people  as 
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well  as  with  an  inanimate  product.  A  review  is  not  an  inquisition 
and  is,  in  fact,  nearly  always  best  conducted  as  a  presentation 
by  the  responsible  engineers  to  the  review  group.  Many  contrac¬ 
tors  have  found  that  good  engineers  welcome  it  as  an  opportunity 
to  demonstrate  capability  before  their  peers  and  contemporaries. 

1.  PHASES  OF  DESIGN  REVIEW 


Four  phases  of  design  review  are  readily  identifiable. 

1.1  CONCEPTUAL  PHASE  OF  DESIGN 

During  conceptual  design,  the  designer  will  want  to  take  advan¬ 
tage  of  all  of  the  previous  experience  and  information  that  he 
can  possibly  make  available  to  himself.  Some  of  the  types  of 
information  that  he  will  want  to  consider  are  (a)  what  is  the 
experience  of  similar  equipment;  (b)  the  failure  reports  on  un¬ 
satisfactory  conditions  of  similar  designs;  and  (c)  test  inform¬ 
ation  and  technical  engineering  papers  reporting  on  experience 
of  other  designers  even  to  the  extent  of  examining  designs  in 
other  industries. 

w 

During  this  phase,  numbers  of  different  ideas  are  generated, 
modified,  and  discarded  for  various  reasons.  They  should  be 
studied  in  a  series  of  successive  approximations  before  one  is 
selected  for  detailing.  Many  times  after  extensive  layout  and 
initial  detailing  is  started,  or  even  when  complete,  a  deficiency 
will  be  uncovered  which  requires  throwing  the  whole  idea  out  and 
starting  over.  Sometimes  a  further  look  at  one  of  the  early 
ideas  previously  discarded  will  offer  a  solution  for  final  de¬ 
sign.  It  is  here  that  system  modeling  techniques  are  of  greatest 
value.  The  more  complete  the  technical  evaluation  at  the  con¬ 
ceptual  stage,  the  more  probable  that  time  will  ultimately  be 
saved,  and  the  design  will  survive.  Initial  dependability  re¬ 
quirements  are  established  and  apportioned  for  compliance  at 
this  stage. 

Once  the  designer  has  gathered  all  of  his  information  described 
above,  he  will  want  to  review  it  prior  to  proceeding  on  to  the 
next  phase  of  design.  To  most  effectively  accomplish  this,  he 
will  need  to  hold  a  discussion  with  many  participants  from  whom 
he  has  gathered  the  information  during  the  conceptual  design. 

During  the  conceptual  design  stage,  Design  Reviews  usually  con¬ 
sist  of  informal  meetings  held  between  the  designer,  his  immed¬ 
iate  supervisor,  the  section  chief,  and  possibly  with  participa¬ 
tion  at  higher  levels  of  the  organization  when  ths  importance  of 
the  design  warrants. 
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At  the  start  of  a  difficult  or  major  development  program,  it  may 
be  desirable  to  extend  the  scope  of  and  participation  in  these 
meetings  by  conducting  formal,  fully  documented  meetings  utiliz¬ 
ing  the  services  of  other  well  qualified  members  of  the  crgani- 
zation  or  outside  consultants  in  certain  specialties.  Such 
meetings  are  generally  called  Formal  Design  Reviews. 

Informal  meetings  generally  are  brainstorming  sessions  for  the 
purpose  of  generating  new  ideas  and  to  help  the  designer  to 
develop  an  awareness  of  various  problems,  as  well  as  to  solidify 
design  ideas.  From  these  reviews,  the  designer  develops  suf¬ 
ficient  confidence  to  proceed  further  into  the  Preliminary  design 
phase. 


1.2  PRELIMINARY  DESIGN 


Once  the  conceptual  design  review  has  been  completed,  the  next 
step  is  the  preliminary  design.  A  layout  is  now  required  to 
determine  how  to  install  or  assemble  equipment  into  its  parti¬ 
cular  area.  To  accomplish  this,  the  designer  lays  out  the  parti¬ 
cular  area  as  near  to  full  scale  as  pos^  le.  To  assist  the 
designer  in  visualizing  how  various  designs  will  appear  in  a 
third  dimension  and  also  as  an  eff^fctive  coordination  tool  with 
other  system  designers  working  on  different  functions,  a  mock-up 
should  be  utilized.  The  chief  designer  of  an  aircraft  company 
referred  to  this  as  "a  three  dimensional  layout  for  the  designer." 
The  mock-up  at  this  particular  stage  should  be  very  flexible  in 
order  that  the  designer  can  quickly  get  different  ideas  mocked 
up  and  be  able  to  investigate  many  possibilities  in  a  design  for 
a  given  period  of  time. 

The  informal  reviews  continue  on  through  the  preliminary  design 
phase  to  assist  the  designer  in  meeting  milestones,  staying 
within  the  budget,  and  arriving  at  a  balanced  design.  When  the 
preliminary  design  is  complete,  a  Formal  Design  Review  is  held. 

1,3  FORMAL  DESIGN  REVIEW 

A  great  deal  of  organization  and  technical  effort  is  necessary 
to  provide  the  basic  framework  necessary  for  successful  design 
reviews.  Several  principles  have  proven  to  be  important  regard¬ 
less  of  specific  program  details* 

a.  The  efficiency  of  design  reviews  is  a  direct  function  of 
the  effectiveness  of  the  communi cation  techniques  used 
between  project,  personnel^  designers,  reliability,  com¬ 
petent  specialists,  etc. 
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b»  Design  reviews  must  be  conducted  in  accordance  with  pro¬ 
gram  milestone  schedules  and  specific  ground  rules,  a 
checklist  or  its  equivalent  must  be  used  to  assure  adequate 
consideration  of  elements  such  as  reliability  vs.  reliabil¬ 
ity  requirements,  and  maintainability  vs.  maintainability 
requirements . 

c.  Design  reviews  must  be  to  the  point,  brief,  must  not  drift 
away  from  the  topic  under  consideration,  must  be  confined 
to  essential  personnel,  must  not  include  "interested" 
personnel. 

d.  Design  reviews  must  make  provisron  for  corrective  actions 
to  be  identified  and  monitored. 

e.  Design  reviews  must  be  adequately  funded  and  data  must  be 
recorded  to  assure  that  the  results  can  be  evaluated  at 
some  future  ciate. 

Every  project,  every  design  for  that  matter,  is  evaluated  with 
regard  to  its  ultimate  function,  reliability,  and  maintainability. 
Whenever  one  circuit  is  selected  over  another  or  a  part  or  a 
packaging  configuration  is  chosen,  the  engineer  has  weighed  some 
trade-offs  and  has  made  a  decisiorr  as  to  the  adequacy  of  one 
design  versus  another. 

When  actions  such  as  these  are  performed  on  the  basis  of  minimum 
information  or  is  the  result  of  empirical  techniques  which  have 
evolved  through  the  years  there  is  a  strong  possibility  for  the 
creation  of  inadequate  designs,  costly  errors  and  incompatibility 
as  various  segments  of  an  equipment  are  integrated.  Through 
design  reviews  we  are  able  to  assure  ourselves  of  a  uniform  high 
quality  of  designs,  even  though  hundreds  of  personnel  are  involved. 
The  standards  remain  in  the  hands  of  a  very  few  people.  Even 
though  most  design  reviews  do  not  have  the  authority  to  reject  a 
design,  their  influence  is  felt  very  strongly  in  the  adhetence  to 
these  standards . 

Mutual  exchange  of  technical  information  is  of  great  advantage  to 
program  efficiency  in  product  design.  This  cross- fertilization 
of  design  techniques  through  the  medium  of  a  design  review  improve 
the  capability  of  all  engineering  personnel  participating.  The 
properly  organized  design  review  program  utilizes  the  available 
capacity  of  specialists  in  an  optimum  fashion. 

Contractually  there  are  many  difficulties  most  of  Which  must  be 
overcome  in  order  to  perform  in  accordance  with  contractual  re- 
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qmrements .  Design  reviews  are  almost  a  necessity  in  dealing  with 
subcontractors  or  associate  contractors,  particularly  for  the  prime 
or  the  lead  contractor.  The  monitoring  of  design  reviews!  is  difficult 
and  yet  in  order  to  have  an  integrated  weapon  system  it  must  be 
done.  Few  companies  are  willing  to  send  copies  of  their  design 
reviews  outside  their  organizations.  This  is  tantamount  to  tell¬ 
ing  the  competitor  what  kind  of  problems  they  have.  It  is  a 
private  problem  and  this  difficultymust  be  overcome  when  partici¬ 
pating  as  a  subcontractor  or  a  co- contractor. 

The  primes  must  review  the  subcontractor  and  frequently  must  con¬ 
duct  a  design  review  of  the  subcontractor's  product  themselves. 

For  maximum  system  worth  the  design  and  the  reliability  efforts 
of  all  organizations  must  be  compatible.  The  activities  of  com¬ 
panies  with  widely  varying  policies  and  design  approaches  must 
be  integrated.  In  this  type  of  effort  the  responsibility  of  the 
prime  contractor  is  to  coordinate,  st  ...ciardize  designs,  finishes, 
parts,  components,  reporting  methods  and  frequently  control  the 
level  of  the  engineers  used  on  the  job. 

A  subject  which  will  come  under  frequent  discussion  is  Bureau 
personnel  attendance  at  contractor  design  reviews.  The  arguments 
presented  in  the  preceding  paragraph  apply  to  this  case  too. 

Bureau  engineers  must  realize  that  their  attendance  will  tend  to 
limit  the  free  exchange  of  information  and  the  open  expression 
of  divergent  opinions  on  the  part  of  contractor  personnel.  This 
will  very  likely  result  in  the  design  review  program  being  com¬ 
pletely  ineffective.  However,  cases  may  arise  in  which  the  Bureau 
has  had  a  direct  contact  with  the  design  process  and  is  hence  in 
a  position  to  contribute  significantly  to  the  design  review  pro¬ 
cess.  In  such  cases  Bureau  participation  can  be  justified  and 
is  warranted.  A  good  ground  rule  to  be  followed  could  be: 
attendance  as  an  "interested  party  or  "observer"  should  be  dis¬ 
couraged,  attendance  as  an  active  participant  should  be  encouraged. 

But  if,  as  one  service  has  done,  the  Bureau  wants  to  renegotiate 
contract  dollars  because  of  design  effort  scrapped  as  result  of 
Design  Review,  further  reviews  with  Bureau  participation  will  be 
sheer  white-wash. 

The  primary  difference  in  the  informal  meetings  and  the  Formal 
Design  Review  is  that  the  Formal  Review  formulates  more  definite 
decisions.  Also,  aspects  outside  of  Engineering  are  reviewed 
by  specialists  in  Manufacturing,  Tceling,  Planning,  Logistics, 
Purchasing,  Facilities,  and  Quality  Assurance.  This  provides  the 
Designer  with  assurance  that  his  design  is  progressing  along 
practical  lines.  It  is  to  provide  assurance  that  the  design  can 
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be  (a)  manufactured  economically,  (b)  the  equipment  which  lias  to 
be  obtained  outside  of  the  jmpany  can  be  purchased,  (c)  the  design 
can  be  serviced,  (ci)  can  be  tested  in  production  and  at  Receiving, 
and  (e)  can  be  provided  with  a  yuality  Assurance  Program.  Luring 
this  review  the  information  will  be  documented  and  maintained  as 
a  permanent  record,  due  various  specialists  mentioned  above  form 
a  Design  Review  Board.  This  Board  is  on  call  by  its  chairman. 

If  an  effective  job  is  to  be  done,  it  makes  little  difference 
where  in  the  organization  the  design  review  board  members  report. 
The  essential  requirements  arc  that  the  board,  as  such,  must: 

a.  Report  to  a  manager  other  than  those  whose  designs  are 
reviewed.  No  one  can  effectively  audit  his  own  work. 

Thus  it  may  be  established  within  the  design  assurance 
(related  to  engineering)  area  or  the  product  assurance 
(reliability,  quality,  etc.)  organization  under  general 
management.  Or,  it  may  be  located  within  a  design  assurance 
(reliability,  etc.)  organization  in  Engineering,  if 
separated  from  design  responsibility. 

b.  Report  high  enough  to  attract  and  satisfy  truly  experienced 
engineers  as  review  chairmen  and  participants. 

c.  Have  top-calibre  men.  Review  chairmen  must  be  technically 
experienced,  diplomatic,  able  to  re  ignize  "snow",  tough 
when  the  occasion  needs  it,  and  widely  respected.  Men  of 
this  calibi  are  available  in  most  companies,  but 
command  excellent  salaries. 

A  general  FLOW-DIAGRAM  is  shown  in  Figure  15-8  for  a  Design 
Review  System.  When  the  Formal  Reviews  are  complete,  the  designer 
can  now  accomplish  several  steps  toward  completing  his  design  and 
providing  for  the  manufacturing  of  the  design.  Some  of  these 
steps  are  (a)  write  advanced  material  orders  for  both  material 
and  parts;  (b)  release  information  for  a  mock-up;  (c)  write  some 
of  the  test  plans  and  procedures;  (d)  order  new  facilities  that 
may  be  required  for  manufacturing  and  tests;  (e)  provide  inform¬ 
ation  to  Program  Evaluation  and  Review  Technique  (PERT,;  (f)  in¬ 
form  other  designers  who  are  depending  on  the  progress  of  the 
design  with  up-dated  information. 

1.4  *-TNAL  DESIGN 

With  Preliminary  Reviews  now  completed,  the  designer  can  proceed 
to  the  rinal  design  phase  which  will  develop  the  details  required 
to  make  the  components  subsystems  or  system. 
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'  r^parati  n  f  >r  a  design  rcvif*  starts  with  the  a^aerbly  f  a 
data  package  on  the  design  approximately  two  weeks  prior  to  the 
Final  Design  Review  of  the  design.  Vais  enables  members  of  the 
board  to  become  familiar  with  the  design  prior  to  the  meeting 
and  to  formulate  comments  on  the  design  to  present  at  the  Design 
Review.  The  major  input  to  the  data  package  must  come  from  the 
designer.  Two  important  benefits  accrue  at  this  point.  (a)  the 
designer  must  make  a  ; ather  thorough  review  of  his  design  in  the 
process  of  preparing  this  material  and  (b)  board  members  who  have 
made  prior  input  to  the  designer  during  the  design  process  will 
check  the  validity  of  interpretation  by  the  designer  and  the 
correctness  of  the  information  transmitted. 

The  Design  Review  Board  should  be  headed  by  an  experienced  chair¬ 
man.  He  calls  the  meeting,  and  conducts  it.  He  asks  the  design 
engineer  to  describe  the  reasoning  behind  his  design,  and  parti¬ 
cularly  behind  unusual  features  and  potential  trouble  areas. 

He  then  asks  for  team  attention  sequentially  to  each  agenda  item. 
He  runs  through  previously-selected  items  of  the  pertinent  check - 
hsts ,  inviting  comment.  When  design  "soft-spots"  are  detected, 
he  leads  the  discussion  to  bring  out  all  viewpoints.  He  then 
expresses  the  team  concensus  of  opinion,  adjusts  it  until  accep¬ 
table  all  around,  and  records  it.  Dissent,  if  any  (it  is  rare  if 
the  team  is  competent),  is  recorded. 

When  a  change  is  recommended,  the  design  engineer  is  asked  what 
action  he  will  take.  In  most  instances,  he  can  respond  on  the 
spot.  But  if  necessary,  the  engineer  may  take  up  to  ten  days 
to  get  more  information,  and  is  required  then  to  respond. 

The  formal  meeting  is  then  closed,  and  a  report  issued.  The 
report  specified  - 

1.  Areas  of  outstanding  design. 

2.  What  changes  are  recommended. 

3.  Why  each  change  is  recomme  *  4- 

4.  Who  is  askeo  to  make  each  cnange. 

5.  When  each  change  should  be  completed. 

One  of  the  most  effective  techniques  for  following  up  action  on 
recommended  changes  is  a  Corrective  Action  Log.  This  names  the 
department  and  individual  responsible,  the  design  identification, 
the  action  to  be  taken,  the  status  of  the  action,  the  scheduled 
completion  date,  and  the  estimated  completion  date.  An  audit 
group  controls  this  log,  reissues  it  weekly,  and  makes  sure  u  at 
names  stay  on  the  list  until,  action  is  completed  by  all  concerned. 
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When  action  drags,  a  marked  copy  is  sent  to  the  supervisor  of 
the  man  v ho  should  act. 

When  the  final  layout  has  been  completed,  a  Formal  Design  Review 
is  conducted  in  the  same  manner  a*  described  for  the  preliminary 
design  but  the  review  is  oriented  more  toward  the  design  details 
rather  than  the  system  conception.  An  agenda  is  again  used.  A 
much  more  specific  review  can  now  be  made.  This  review  assists 
the  designer  with  information  with  which  to  evaluate  and  direct 
the  detailed  implementation  of  the  design.  That  is,  the  review 
can  consider  parts  applications,  tolerance  analysis.  Reliability 
and  Maintainability  prediction  vs.  requirements,  emphasis  of 
certain  dimensions,  the  need  of  production  tests,  the  type  of 
material  processing,  the  assembly  sequence,  the  areas  of  Quality 
Engineering  emphasis  and  the  schedule  required  on  various  parts 
for  tests.  Also,  certain  deviations  from  the  customer's  speci¬ 
fications  may  be  required.  In  order  to  avoid  delay  when  equip¬ 
ment  is  completed,  deviation  requests  must  be  submitted  early 
for  customer  approval. 

The  designer  then  proceeds  to  direct  the  completion  of  the  design. 
Variations  determined  in  the  drawing  'of  the  details  need  to  be 
noted  on  the  layout  as  well  as  the  part  number  of  the  detail. 

Also,  a  notation  of  the  analytical  record  and  reports  is  noted 
on  the  layout.  Experiences  have  shown  the  well  documented  layc ut 
is  invaluable  when  failures  occur  and  corrective  action  is  sought. 
The  layout  provides  a  central  source  of  information  and  this 
enables  the  designer  to  quickly  evaluate  the  background  and  saves 
much  time  arriving  at  a  solution  to  a  failure. 

Final  Design  Review  is  now  in  order  to  evaluate  the  design,  with 
the  assistance  of  all  of  the  knowledge  possible  prior  to  its 
final  use.  A  checklist  gives  an  idea  of  the  considerations  which 
must  be  made  at  this  time.  The  various  specialists  on  the  Design 
Review  Board  assist  the  designer  materially  in  providing  inform¬ 
ation  which  finally  can  be  best  evaluated  by  the  designer  who 
knows  the  very  "heart-beat"  of  the  design.  With  this  detailed 
information  supplied  by  the  Design  Board,  he  can  now  make  the 
final  evaluation  of  the  end  item.  This  is  the  last  opportunity 
for  the  designer  to  assure  that  his  design  will  be  successful  as 
originally  released. 

1.5  ALTERATION  OR  CORRECTIONS 

When  design  changes  are  needed  after  initial  release,  design  re¬ 
view  is  of  equal,  if  not  more,  significance  than  at  previous  times. 
The  plan  alterations,  revisions  and  fixes  are  the  indication  of 
hiddfen  problems.  Because  of  their  nature  they  require  the  most 
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rigorous  attention  in  design  review  for  permanency  of  adequate 
solution.  They  are  also  the  most  costly  in  time  and  dollars, 
since  they  represent  " re-do"  of  both  design  and  finished  manu¬ 
facture,  as  well  as  requiring  field  changes  and  logistic  cost  if 
delivered. 

In  this  area,  the  pressures  on  the  designer  to  hurry  the  fix, 
obviate  the  onus  of  mistakes,  release  new  material,  recover  from 
schedule  delinquencies,  etc.,  creates  an  environment  that  leans 
more  to  "get  it  fixed  and  out  at  any  cost"  than  to  considerations 
of  the  reliability;  i.e.,  permanent  adequacy  of  the  fix.  These 
pressures  lead  to  relaxation  of  taking  technical  advantage  of 
the  lessons  learned.  Formal  Review  of  the  design  must  be  re¬ 
quired  for  any  design  change. 


2.  CHECKLISTS 

.  .f  V  . :  ‘  '  *••••'* 

Two  types  of  checklists  are  used  in  conducting  design  reviews! 

(a)  a  design  review  checklist  for  use  in  preparation  .of  the  data 
package,  agenda,  and  reports,  and  (b)  technical  lists  for  parti¬ 
cular  types  of  analyses.  The  completed  agenda  serves  as  a  Av 
checklist  for  the  design  review  meeting  and  the  report. 

... ..  )  * 

The  following  paragraphs  give  examples  of  each. 

2.1  DATA  PACKAGE  CHECKLIST  A 

1.  List  all  required  functions  of  the  component,  system  or  pro¬ 

blem,  including  allowable  performance  ranges,  variabilities ,  and 
parametric  variations.  , 

2.  List  all  environmental  conditions  by  magnitude  and  frequency, 

including  transients  which  the  component  must  withstand  in  test¬ 
ing  and  curing  service  life.  ,  n  Ai’Sl 

3.  List  all  materials  to  be  used  in  the  system,  component  or 

part,  with  the  properties  of  each  under  the  environmental,  con-^  | 
ditions  expected.  List  the  variability  of  these  properties.  ;  ' 

4.  Outline  complete  test  plan  including  requirements  for'  any  - 
special  test  equipment. 

2.2  GENERAL  DESIGN  CHECKLIST  A  .  '  -1  > 

1.  Review  all  basic  parameters  included  in  the  data  package  for ; 
correctness  and  completeness.  ^ 

».  .  >•-.  ■  fb.'  ■:  r  .. 


2.  Examine  the  sub  }ect  design  ivr  component  to  determine  if  pro¬ 
visions  for  each  functional  requirement  have  been  included  in  th 
design.  Establish  the  feasibility  of  h  'Id ing  these  to  specified 
variability  in  manufacture  and  define  the  level  of  confidence 
that  must  be  generated  to  assure  that  the  variability  is  within 
limits . 

3.  Mote  ar'v  capabilities,  features,  accuracies  or  specified 
tests  which  are  beyond  the  s late-of-the-art  or  beyond  the  func¬ 
tional  capabilities  of  the  design  facilities. 

4.  Examine  the  design  approach  to  determine  if  the  simplest 
possible  means  for  obtaining  the  required  function  has  been  de¬ 
veloped.  Reliability  varies  inversely  with  some  power  of  the 
number  of  parts  used. 

5.  Determine  if  proven  (by  test  or  similar  application  history) 
components  and  parts  have  been  used  wherever  feasible. 

6.  vh.eck  the  stress  analysis  (including  structural)  of  each 
component  or  part,  and  determine  critical  loading  and  possible- 
failure  modes.  Look  for  points  of  stress  concentration  or  in¬ 
tensifications,  and  other  possible  weak  lines  (including  limit  in 
parameters  to  continuity  of  performance).  Look  for  load  concen¬ 
trations  due  to  externally  applied  shock?  and  for  noise  contri¬ 
butions  . 

7.  Compare  the  resistive  strengths  (and  any  established  allow¬ 
ables)  of  each  material,  with  the  calculated  load  stresses 
expected.  Indicate  the  ranges  of  variability. 

8.  Examine  the  possibility,  and  the  effects  of  deflection  under 
load,  of  each  component,  or  part,  on  the  performance  required 
from  it  function.  Estimate  external  shock  effects  and  resonant 
vibrations  on  performance  and  life  expectancy. 

9.  Determine  the  compatibility  of  the  material  and  finishes  wiv 
eat'  other,  in  assembly,  under  the  expected  environments.  If 
data  is  net.  available,  estimate  testing  requirements. 

10.  Consider  the  possibility  and  effects  of  predictable  wear 
on  the  maximum  allowable  tolerances,  as  related  to  the  perform¬ 
ance  factors  of  the  components. 

Ll.  Consider  the  possibility  and  the  effects  of  adverse  toler¬ 
ance  build1’^  >n  each  part,  including  the  effects  of  thermal  ex¬ 
pansion,  vi  ation,  and  differential  shock  excursions. 
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’.2.  Consider  the  producibi 1 lty  of  each  component  or  part  under 
the  manufacturing  conditions  in  which  it  will  be  built. 

13.  Consider  the  related  aspect?  of  accessibility,  repairabi lity , 
maintainability  (including  lubrication)  and  operability  under 
field  conditions  with  the  variabilities  of  skill  and  morale  of 
personnel . 

14.  Consider  the  convenience,  special  tools  and  accuracy  re¬ 
quired  for  operational  adjustments  and  control  instrumentation, 
from  a  human  factors  standpoint. 

15.  Consider  the  effects  of  associated  random  casualty  and 
permanent  shock  effects  on  the  performance  characteristics  of 
the  total  system. 


16.  Consider  the  compatibility  of  the  components  and  parts  with 
each  other  and  with  supporting  services  in  the  system. 

17.  Consider  the  installation  criteria  (handling,  alignment, 
etc.)  for  the  system,  component  or  part  in  the  overall  arrange¬ 
ment  . 


18.  Review  the  overall  evaluation,  summarize  and  conclude, 
noting : 


a.  The  possible  design  deficiencies,  including  contract  or 
specification  deficiencies  or  conflicts. 

b.  The  probable  and  possible  modes  of  failure  and  the  effect 
of  these  or  both  the  component  and  overall  system. 

c.  The  tests  deemed  necessary  to  establish  data  for  final 
reliability  assurance. 

d.  Any  inspection  procedures  either  routine  or  special, 
which  would  help  uncover  most  likely  manufacturing  and 
assembly  errors. 

e.  The  test  deemed  necessary  to  fully  evaluate  performance 
vs.  design,  failure  modes,  and  overload  conditions. 

f.  For  parallel  components  other  components  that  can  fail 
without  causing  a  detectable  system  malfunction,  list  the 
periodic  inspection  procedures  that  will  monitor  these  poten¬ 
tial  failure  points. 


I 
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g.  The  criteria  (including  time)  for  periodic  preventive 
maintenance  and  repair. 

h.  The  important  operational  cautions  which  should  be  included 
in  instruction  books  and  maintenance  manuals. 


i.  Review  life  expectancy  data  and  associated  parametric 
criteria  for  failure  vs.  time.  List  suggested  service  life 
periods  to  component  overhaul  an'  or  replacement. 

j.  Review  all  data  developed  and  fai lure/ time  information  on- 
similar  components.  Estimate  a  mean  time  to  first  failure 
and/or  the  safety  margins  available  to  assure  compliance  with 
contract  requirements. 

K.  The  developed  information  on  probable  life  expectancies 
assuming  adequate  maintenance  and  repair  performance.  Esta¬ 
blish  level  of  essentially  and  list  all  inputs  for  estab.i  '  ^ fl¬ 
ing  confidence  in  predictions. 

1.  List  agreed  actions  to  be  taken  by  functional  ine  depart¬ 
ments  in  connection  with  de f iciencies ,  errors,  or  inadequate 
raethods/procedures .  Establish  commitments  and  follow-up. 

m„  Write  a  factual  report. 

2.3  STRUCTURAL  FATIGUE  CHECKLIST 

1.  Was  major  attention  given  to  actual  stresses,  especially  at 
stress  concentrat ions ,  rather  than  to  the  nominal  average  stresses? 

2.  Did  you  visualize  how  load  is  transferred  from  one  part  or 
section  to  another  in  a  structure  and/or  the  distortions  that 
occur  during  loading,  to  help  locate  the  points  of  high  stress? 

3.  Were  gradual  changes  in  section  and  symmetry  of  the  design 
used  according  to  such  design  criteria  as  shown  in  Section  XX  of 
the  Design  Manual? 

4.  Was  careful  attention  given  to  location  of  joints  and  type 
of  joints  used?  (Joints  are  one  of  the  most  frequent  sources  of 
fatigue  weakness.) 

5.  Were  symmetrical  joints  used  wherever  possible? 


6.  Were  suitable  means  used  to  stiffen  unsymmetr ical  joints  so 
that  secondary  flexing  is  reduced  to  a  minimum? 


i 
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7.  Did  you  design  joints  so  that  all  parts  will  participate 
equally,  and  tnat  there  will  not  be  an  undesirable  load  transfer 
to  an  adjacent  part? 

8.  Did  you  avoid  open  hoi  ns  and  loosely  filled  ones? 

9.  Was  preference  given  in  the  design  to  butt  jcdnts,  as  detailed 
in  the  Design  Manual,  Section  IV,  Chapter  5? 

10.  Did  you  give  preference  to  redundant- type  structures  where 
this  type  of  structure  is  possible? 

11.  Was  careful  attention  given  to  fabrication  details  to  im¬ 
prove  fatigue  life? 

12.  Were  the  proper  surface  finishes  chosen? 

13.  Was  suitable  protection  against  corrosio...  provided? 

14.  Was  attention  given  to  the  gcanetry  of  a  welded  joint  in¬ 
cluding  such  factors  as  smoothness,  undercutting,  cracks,  exces¬ 
sive  porosity,  spatter,  and  symmetry? 

15.  Did  you  design  for  accessibility  for  inspection  of  important 
tension  joints? 

16.  Was  the  addition  of  secondary  brackets,  fittings,  handles, 
steps,  bosses,  grooves,  and  openings  at  locations  of  high  stress 
avoided? 

17.  Is  part  material  compatible  with  its  function  and  loading? 

18.  Is  the  type  of  construction  best  suited  for  the  loading 
conditions  including  sonic  fatigue? 

19.  Are  the  unsupported  panel  sizes  small  enough  to  resist  sonic 
fatigue ? 

20.  has  maxirum  simplicity  been  achieved  consistent  with  sonic 
fatigue? 

21.  Is  *he  part/assembly  sensitive  to  fatigue  from  ground  hand¬ 
ling  an  vibration  to  be  encountered? 

22-  Are  there  any  unnecessary  joints  and  splices? 


23.  Are  there  any  possibilities  of  chain  reaction-type  failures 
which  can  be  prevented? 
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24.  do  the  fastener  selections  best  satisfy  all  load  require¬ 
ments  ? 

25.  Are  fastener  bolt  sizes  and  tolerances  compatible  with  part 
functions  ? 

26.  Has  the  proper  bolt  torque  been  specified? 

27.  Have  retaining  or  locking  rings  been  eliminated  where 
possib] e? 

28.  Have  the  heat  treat  steels  been  considered  for  hydrogen 
embr  it  t  lenient  ? 

29.  Has  Design  Manual  plating  practice  been  adopted? 

30.  Has  ample  clearance  been  allowed  for  structural  deflection 
of  adiacent  parts? 

31.  Has  effect  of  structural  deflection  of  one  part  on  others 
attached  to  it  been  considered? 

32.  Have  thermal  stresses  and  differential  thermal  expansion 
been  considered? 

33.  Have  the  following  sources  of  stress  concentration  been 
eliminated? 

a.  Sharp  corner  and  fillet  radii? 

b.  Eccentric  load  paths? 

c.  Abrupt  section  changes? 

d.  Stiffeners  terminating  in  middl  of  unsupported  panels? 

e.  Clip  angles  attached  to  web  only? 

f.  Steel  stamp  part  numbering  in  areas  of  high  stress? 

34.  Is  secondary  structure,  wh ; ch  ic  rigidly  attached  tc  primary 
structure,  designed  to  carry  the  loads  induced  in  it  by  deflec¬ 
tion  of  the  primary  structure? 

35.  Has  the  primary  structure  beer,  reviewed  for  advers  1  load 
distribution  caused  by  secondary  structure? 
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2.4  mMAg  factors  checkl ist 

I . .  Has  the  best  allocation  of  function  between  man  and  machine 

been  determined? 

2.  Have  the  controls  and  indicators  been  designed  and  arranged 
with  body  measurement  limitations  duly  considered? 

3.  Has  the  location  of  indicators  and  controls  been  balanced 
against  the  need  for  adjustment? 

4.  Has  consideration  been  given  to  the  use  the  operator  will 
make  of  each  instrument,  control,  and  equipment  and  as  to  how  its 
location  will  aid  in  the  performance  of  the  task  with  the  most 
accuracy  and  least  fatigue? 

5.  Has  the  type  of  response  he  must  make  been  determined? 

6.  Will  present  design  interfere  with  his  ability  to  continue 
receiving  the  information  he  needs? 

7.  Is  speed  cf  operation  critical? 

8.  Is  accuracy  of  reading  or  setting  critical? 

y.  Have  the  job  operations  been  simplified  to  present  to  the 
operator  the  fewest  possible  motions,  the  nature  sequence  of 
motions  and  only  pertinent  information,  in  order  to  minimize  the 
chance  of  failure  under  stress? 

10.  May  adjustments  and  alignment  be  accomplished  by  an  average 

technician? 

II.  Has  consideration  been  given  to  the  operators  psvchological 
and  environmental  conditions  during  the  operation  of  the  equip¬ 
ment? 

12.  Have  the  different  kinds  of  illumination  been  considered? 

13.  Have  glare  hazards  been  eliminated,  such  as  brightly  polished 
bezels,  glossy  enamel  finishes,  or  highly  reflective  instrument 

covers  ? 

14.  Have  the  static  dimensional  data  for  cabinets,  racks  and 
consoles  been  used  in  the  design  with  the  dynamic  dimensional 
statistics  of  the  human  operator  in  mind? 
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15.  Are  vertically  mounted  visual  displays  50  to  70  inches  above 
the  floor  when  they  are  to  be  viewed  for  a  standing  position? 

16.  Has  a  30-inch  seat-to-eye  height  reference  been  used  to 
locate  visual  displays  for  a  seated  operator  and  has  the  chair 
height  been  specified  along  with  the  console  dimensions? 

17.  For  a  comfortable  display  mounting  angle,  has  the  following 
rule  been  used:  60  degrees  from  horizontal  for  seated  operator 
position,  45  degrees  for  a  combination  position  of  sitting  or 
standing  and  30  degrees  for  a  standing  position? 

18.  Has  a  28-inch  arm  reach,  measured  from  the  operator's 
shoulder,  been  used  as  a  limiting  figure  for  the  placement  of 
controls  which  are  to  be  used  often? 

19.  Have  the  controls  been  located  near  the  display  which  they 
affect  when  this  does  not  conflict  with  other  manipulatory  re¬ 
quirements  ? 

20.  Have  the  controls  been  arranged  sequentially  with  respect 
to  be  expected  or  required  order  of  operation? 

21.  Do  arrangement  and  layouts  stress  the  importance  of  balanc¬ 
ing  the  workload  or  do  they  force  one  hand  to  perform  too  many 
tasks  while  the  other  hand  is  idle? 

22.  Considering  functional  requirements,  is  the  panel  layout  as 
simple  as  practicable. 

23.  Are  interdependent  functions  so  arranged  that  adjustment 
and  troubleshooting  are  amenable  to  logical,  straight  forward 
procedures  ? 

24.  Do  visual  displays  occupy  central  areas  and  controls  occupy 
peripheral  areas  whenever  possible  to  avoid  hand  and  aj.m  inter¬ 
ference  with  visual  tasks? 

2.5  DEVELOPMENT  OF  CHECKLISTS 

It  is  apparent  that  the  checklists  should  be  prepared  by  the  man 
or  group  most  competent  in  the  specialty.  In  the  conduct  of  the 
meetings  a  complete  agenda  should  Lrt  used  to  assure  nothing  is 
omitted.  Points  of  no  consequence  Cc.n  be  summarily  discussed 
and  disposed  of.  The  technical  checklist  should  be  prepared  by 
an  experienced  specialist  who  contributes  his  knowledge  of 
pattern  c  f  error  he  has  discerned  in  his  experience  with  a  wide 
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variety  of  designs. 

The  technical  checklists  should  be  furnished  to  the  designer  in 
the  very  early  stages  of  design.  They  should  be  devised  for  con¬ 
venient  use  during  the  design  phase  and  should  be  required  to  be 
submitted  to  his  supervisor  with  the  completed  design,  along  with 
analyses  and  other  documentation. 


3.  COVERAGE  OF  DESIGN  REVIEW 

It  is  obviously  impractical  and  inefficient  to  conduct  separate 
reviews  of  Reliability,  Maintainability,  Producibility ,  Testa¬ 
bility  and  the  dozens  of  other  disciplines.  The  design  review 
should  comprehend  all  of  the  factors  that  make  a  good  design. 

In  the  paragraphs  that  follow,  some  of  the  aspects  will  be  covered. 

3 . 1  MAINTAINABILITY 


One  of  the  major  areas  of  review  is  the  maintainability  of  the 
product.  The  design  must  be  subject  to  review  by  those  who  plan, 
design  and  have  the  responsibility  for  support  of  the  weapon 
during  its  operational  life.  This  is  accomplished  primarily  by 
a  maintenance  engineering  analysis  of  the  end  article,  systems 
and  components  thereof  by  a  group  external  to  the  design  engin¬ 
eering  department. 

Successful  implements  on  of  a  competent  maintenance  engineering 
analysis  (MEA)  requires  management  support,  proper  funding,  pro¬ 
per  planning  and  most  important,  personnel  with  the  proper  back¬ 
ground  and  training.  The  MEA  changes  recommended  must  be  con¬ 
sidered  from  all  aspects  including  necessary  tradeoffs  of  cost, 
weight,  performance  and  mission  accomplishment.  Thus  the  MEA 
must  be  a  part  of  the  design  review. 

To  be  effective,  the  MEA  should  be  initiated  during  the  proposal 
stage  and  carried  forward  during  drawing  layout,  drawing  release, 
parts  manufacture,  and  assembly  and  test  of  components  and  the 
end  article. 

The  maintenance  analyst  must  know  the  maintenance  level  at  which 
each  bit,  piece,  part,  component,  system  or  end  article  will  be 
serviced,  repaired  or  overhauled.  The  analyst  must  establish  the 
detailed  maintenance  tasks  required  to  maintain  the  end  article 
in,  or  return  it  to  a  mission  ready  status.  The  analyst  must 
justify  the  tools  and  test  equipment  required  to  accomplish  each 
task.  He  must  source  code  the  parts  required  to  accomplish  each 
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task  and  assign  procurement  factors  as  to  quantities  required. 

Ke  must  be  able  to  identify  any  new  special  skills  required  to 
perform  the  tasks  and  determine  that  such  skills  will  be  avail¬ 
able  at  the  using  activity.  He  must  determine  that  the  necessary 
technical  instructions  will  be  available  to  the  man  performing 
the  task. 

3.2  PARTS  CONTROL 

One  of  the  major  causes  of  unreliability  is  the  use  of  unreliable 
parts.  The  designer  has  a  penchant  for  using  new  and  novel  ideas 
based  on  apparently  outstanding  performance  capabilities  seen  in 
reports  or  advertising  without  verification  of  the  actual  per¬ 
formance  '  f  the  part.  Since  parts,  just  like  systems  must  be 
developed,  the  bugs  eliminated  and  the  design  matured  to  reliable 
operation,  such  innovations,  more  often  than  not,  create  relia¬ 
bility  problems.  Parts  specialists  study  the  actual  data  on 
parts  utilization  to  determine  the  degree  of  maturity  of  many 
parts,  and  can  provide  records  of  actual  successful  applications 
of  the  parts  in  many  systems.  Such  a  record  of  satisfactory  use 
provides  some  reasonable  assurance  of  success  in  similar  applica¬ 
tions.  Where  no  evidence  of  successful  use  can  be  found  the 
use  of  t.ie  part  is  suspect.  The  parts  specialist,  in  addition 
to  being  available  for  information  on  past  uses,  should  review 
the  parts  proposed  to  identify  such  suspect  applications. 

3.3  .MANUFACTURING  PROCESS  ENGINEERS 

While  the  designer  is  the  expert  on  the  requirements  of  the  final 
product,  he  is  not  necessarily  expert  in  tne  manufacturing  pro¬ 
cesses  necessary  or  useful  in  achieving  such  requirements.  The 
capability  of  the  manufacturing  tools  and  machines  is  the  pro¬ 
vince  of  production  engineers.  These  latter  can  discover  areas 
in  the  design  that  cannot  be  built  within  the  tolerances  assigned 
using  equipment  available  in  the  plant.  They  can  propose  changes 
in  the  specifications  and  drawing  which,  if  acceptable,  will 
permit  accomplishment  with  available  tools,  better  inspection  or 
less  expensive  manufacture.  Their  day  in  court  at  a  design 
review  frequently  improves  the  producibi 1 ity  of  the  product  with¬ 
out  degrauing  (and  often  improving)  performance  and  reliability. 

j  .  4  RELIABILITY  ENGINEERS 

The  position  of  the  Reliability  Engineer  is  that  of  the  critic. 

The  designer  is  primarily  oriented  toward,  seeing  that  the  design 
will  work,  whereas  the  critic,  or  reliability  Engineer,  is  attemp¬ 
ting  to  ferret  out  those  areas  which  would  cause  the  design  not 
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to  operate,  the  indication  of  its  unreliability. 

By  systematic  approaches  of  the  mathematical  model,  failure 
effect  analysis,  anu  criticality  lists,  the  designer  can  be 
materially  assisted  in  arriving  at  the  portions  of  his  design 
that  he  must  concentrate  on  in  order  to  arrr’e  at  a  balanced  and 
reliable  design.  In  a  complex  system,  this  always  is  a  difficult 
assessment  for  the  designer  to  make  without  the  servi ms  of  the 
service  type  of  organizations.  Since  the  mathematicax  model  is 
a  systematic  functional  diagramming  of  components  as  they  fit 
into  subsystems  and  systems,  this  model  can  materially  aid  the 
designer  in  having  an  overall  feel  for  the  complete  system. 

Tnis,  in  turn  will  help  him  in  providing  designs  that  will  pro¬ 
vide  reliability  for  the  total  system  rather  than  overdesign  a 
particular  portion  of  it  and  thereby  not  improve  the  total  relia¬ 
bility. 

3.5  QUALITY  CONTROL  ENGINEERS 

The  assurance  that  the  manufactured  article  conforms  to  the 
specifications  is  the  province  of  Quality  Control.  Their  parti¬ 
cipation  in  design  reviews  will  assist  the  designer  in  making 
sure  that  the  required  quality  is  specified  and  that  the  necessary 
controls  and  inspections  can  be  performed  to  assure  that  it  is 
achieved . 


4.  EFFECTIVENESS 

Fear  of  things  not  understood  is  often  a  constraint  upon  design 
review.  Such  fear  turns  gradually  into  appreciation  once  an 
engineer  realizes  hew  much  he  has  been  helped;  how  he  has  been 
protected  from  the  consequence  ot  mistakes.  The  poor  eng i nee- 
may  risk  exposure  of  his  inadequacies,  but  the  good  engineer 
reaps  the  benefit  of  expert  appreciation  of  his  capabilities. 

Perhaps  the  greatest  single  benefit  of  design  review  is  the 
discipline  of  preparation  for  it.  Analyses  and  tests  are  made 
that  otherwise  might  be  omitted,  checklists  are  checked  and  re¬ 
checked,  and  there  is  more  prior  communication  with  the  special¬ 
ists.  And  this  is  the  path  of  good  design . 

Design  Review  is  a  service  and  audit  for  which  manpower  must  be 
budgeted.  But  the  longer-term  savings  can  far  offset  this  cost. 
To  the  extent  that  design  review  helps  get  work  done  right  the 
first  time,  it  can  minimize  or  eliminate  overrun,  improve  pirofit, 
and  improve  competitive  position.  Specifically: 
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1.  Training :  The  discipline  cl  organized  preparation  for  design 
review,  and  participation  therein,  amounts  to  quicker  on-the-job 
training  of  young  engineers.  Thus  their  learning  curve  is  steeper, 
with  better  efficiency  and  fewer  costl  changes. 

2.  Specialists  :  Fuller  utilization  if  specialist  skills  brings 
earlier  design  adjustments,  avoiding  the  cost  of  later  changes. 

3.  Requirements  Review;  Critical  examination  of  customer  stated 
and  unstated  needs  and  constraints  can  (a)  uncover  unrealized 
requirements  that  otherwise  cause  schedule  slippage  and  unbud¬ 
geted  cost,  (b)  avoid  wasted  design  effort  and  cost  on  non-essen¬ 
tials,  and  (c)  avoid  the  boomerang  cost  of  personal  "understand¬ 
ings"  between  company  and  customer  personnel. 

4.  Pr e f err ad  Components :  Many  parts  and  assemblies  are  useful 
across  many  pro  nets.  The  selection  and  establishment  of  pre¬ 
ferred  components,  for  required  design  use  where  feasible  (a) 
reduces  procurement  cost  through  higher  volume,  (b>  reduces  fac¬ 
tory,  field,  and  customer  inventory  cost,  and  (c)  improves  re¬ 
liability  through  better  but  less  costly  test  and  field  informa¬ 
tion  feedback .  and  higher  reliability  reduces  factory ,  field  and 
customer  costs,  resign  review  assures  the  use  of  preferred  com¬ 
ponents  where  feasible. 

r-.  Value/Cost  :  The  techniques  of  "value  analysis"  have  long 
been  used  for  substantial  cost  reluct  ion  after  release.  The 
same  techniques  can  be  used  it  all  levels  of  the  design  hierarchy 
to  better  approach  minimum  cost  for  the  required  values  before  the 
design  is  laced  ar.  1  rigid  s<»  that  the  opportunity  is  lost.  Also, 
the  cost-effectiveness  techniques  for  system  evaluation,  based 
upon  "reliability"  models ,  provide  a  powerful  tool  for  visibility 
of  consolidated  o  ‘hots  of  proposed  changes.  Effective  design 
l  e  v  i  e w  r  e q  u  ires  thes  e  a r.  •.  lyses. 

6.  Changes :  To  the  extent  th  it  the  above  techniques  detect  the 
need  and  path  f  >r  correction  at  the  earliest  possible  time  during 
design,  they  can  substantially  reduce  the  cost  of  design,  desian 
changes,  redesign,  confusion,  test,  rework,  scrap,  maintenance, 
and  failure  consequence. 
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Chapter  Id 
FAILURE  '  iAGKOSIS 

In  spite  o I  the  care  with  which  the  design  is  performed  and  the 
production  processes  controlled,  only  an  incurable  optimist  will 
expect  a  perfect  product.  As  realists,  we  must  admit  that  mil- 
function  will  occur.  To  provide  for  elimination  of  defects  during 
the  production  and  operational  phase,  every  effort  must  be  made  to 
identify  weaknesses  in  the  product  as  scon  as  they  manifest  them¬ 
selves.  One  of  the  program  elements  leading  to  identification  of 
weaknesses  in  the  product,  and  eventual  correction,  is  the  failure 
reporting  and  analysis  effort. 

i.  CAUSES  OF  FAILURES 

1.1  BASIC  CLASSIFICATIONS 

A  malfunction  of  equipment  may  occur  due  to  any  one  or  a  com¬ 
bination  of  three  basic  causes: 

(a)  Weakness  in  the  design; 

(b)  Error  in  manufacture,  assembly,  inspection  or  testing; 

(c)  Error  by  the  operator  - r  maintenance  mechanic. 

1.2  EXCEPTIONS 


In  addition  to  these  throe  basic  classes,  two  other  types  of  mal¬ 
function  are  reported.  Secondary  failures  are  malfunctions  caused 
directly  or  indirectly  by  the  malfunction  of  an  associated  part  or 
component.  The  failure  should  be  charged  as  a  primary  failure  to 
the  part  nr  component  whose  intial  malfunction  caused  the  secondary 
failure.  In  some  cases  a  reported  failure  cannot  be  confirmed. 

In  many  such  cases  the  report  of  the  failure  was  a  kind  of  operator 
error.  Many  things  can  cause  an  oper  ifor  to  report  a  failure  and 
replace  an  unfailed  item  —  misreading  a  dial,  covering  up  some 
mistake  of  his  own,  being  misled  by  noise  or  vibration  initiated 
somewhere  else.  It  is  important  to  remember  that  not  all  reported 
failures  are  identifiable  or  correct i ble. 

2 .  IDENTIFICATION  OF  CAUSES 

Tue  foundation  of  a  failure  diagn  sis  and  corrective  action  pro¬ 
gram  is  an  effective  failure  reporting  system.  (Figure  16-3) . 

Every  failure,  regardless  of  cause,  should  be  reported.  One 


i 
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procedure,  found  effective  in  industry,  is  to  use  the  failure 
report  form  as  a  requisition  for  new  parts  so  that  the  drawing 
of  a  part  from  stores  automatically  reports  the  expenditure. 

When  this  system  is  used,  obviously,  seme  additional  procedure 
is  necessary  to  obtain  reports  of  failures  repaired  without 
procurement  of  a  new  part.  A  report  should  be  prepared  at  the 
time  of  failure  and  should  include  at  least  the  following: 

(a)  Identi f icat ion  of  the  failed  part.  (Part  name,  part 
number,  serial  number,  identification  of  next  level  of 
assembly) . 

(b)  Identification  of  replacement  part. 

(c)  Operating  life  data  on  the  failed  part. 

(d)  Date  and  activity  area  when  failure  was  discovered 
(Manufacturing,  test  and  operation) . 

(e)  Failure  symptoms.  (Narrative  or  coded  mode  listing  or 
both) . 

( f )  Cause  of  failure,  (narrative  or  coded  mechanism  listing 
or  both) . 

(g)  Action  taken  to  restore  system  to  operation  (i.e.,  re¬ 
placed  part)  . 

Data  collection  systems  are  covered  in  detail  in  Chapter  9. 

2.1  FREQUENCY  UF  OCCURRENCE 

One  of  the  indicators  in  ideat i fientioe  of  a  problem  area  is  the 
excessive  occurrence  of  failures.  A  large  number  of  reported 
failures  on  any  particular  part  number  should  be  investigated  to 
determine  if  a  problem  exists.  The  first  step  is  to  identify  the 
assembly  in  whj.ch  the  failures  are  occurring.  Ii  a  large  number 
of  that  part  number  are  instal1  d  in  different  locations  and  if 
the  failures  are  occurring  at  different  locations  more  or  less 
at  random,  the  part  itself  is  suspect.  If  the  failures  are 
occurring  at  one  or  a  few  of  the  locations,  the  examination 
should  be  made  of  the  system  or  assemblies  in  which  the  parts 
ate  failing. 


Even  a  few  failures  of  <s  particular  part  in  a  particular  assembly 
should  initiate  an  investigation. 


2.2  C IRUUIT  ANALYSIS 


A  reliability  problem  can  be  identified  from  an  excessive  number 
of  failures.  Eve-  a  single  failure  can  indicate  a  reliability 
problem,  however,  when  the  circuit  application  is  such  that  the 
failure  will  prevent  accomplishment  of  a  required  function,  or 
when  safety  of  operating  personnel  may  be  involved. 

When  a  problem  area  is  identified,  the  application  of  the  part 
or  parts  should  be  re-evaluated.  The  circuit  analysis  or  com¬ 
ponent  stress  computation  should  be  reviewed  to  re-establish  the 
designed  stress  levels  (voltages,  pressures,  etc.)  and  the  im¬ 
posed  environment.  The  mechanism  of  failure,  identified  in  the 
failure  report,  should  be  evaluated  against  design  parameters. 
Where  the  failure  appears  improbable  in  light  of  the  stress 
margins  used  in  the  design,  an  extension  of  the  analysis  to 
associated  subsystems,  possibly  even  measurement  of  the  parameters 
in  the  operating  environment  may  be  indicated.  Where  the  cause 
of  failure  cannot  be  identified  from  circuit  or  design  analysis, 
a  failure  analysis  should  be  performed. 

2.3  FAILURE  ANALYSIS 


Failure  analysis  is  the  determination  of  the  cause  cf  failure  or 
equipment  from  test  and  inspection  processes.  Those  component 
malfunctions  that  are  identified  as  problem  areas,  but  not  diagnosed 
from  re-evaluation  of  the  design  are  taken  to  a  testing  laboratory. 
In  the  laboratory,  unless  the  nature  of  the  failure  is  apparent 
from  inspection,  the  equipment  is  set  up  on  test  in  an  assembly 
simulating  its  actual  operational  use.  In  the  test  assembly  it 
is  operated  to  ascertain  the  nature  of  the  malfunction,  and  to 
verify  that  it  is  not  operating  as  designed.  Readings  are  taken 
of  significant  parameters  to  define  exactly  how  the  equipment 
operates . 

Having  verified  the  failure  (or  that  no  apparent  mal function  is 
occurring  in  the  test  set-up)  the  equipment  is  disassembled,  in¬ 
spected  and  measurements  of  important  dimensions  compared  to  the 
drawings.  Where  indicated,  chemical  or  metallurgical  tests  or 
examinations  are  conducted.  As  a  result  of  tests  and  examinations, 
the  analyst  prepares  a  report  covering: 

(a)  Previous  history  of  failures  of  the  part  number  including 
causes  of  failure  ascribed. 

(b)  Significance  factors  in  the  original  report  of  failure 
of  this  part,  including  actual  and  possible  effects  of  the 
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failure . 

(c)  Mode  of  this  failure,  or  report  that  failure  cannot  be 
caused  to  recur. 

(d)  Cause  of  the  failure,  i.e.,  improper  clearance,  corrosion, 
faulty  soldering. 

(e)  Whether  failure  is  classified  as  "design  defect"  or  "not 
in  accordance  wj  th  design  specifications." 

(f)  Recommended  corrective  action. 

Failure  analysis  should  be  continued  until  the  cause  and  means 
of  correction  are  identified.  This  may  require  special  instruct¬ 
ions  to  ship  every  failed  part  to  the  laboratory,  or  trips  by 
analysts  to  the  sites  of  failure,  or  assembly  of  "Tiger  Teams" 
of  designers,  technicians  and  laboratory  people.  Where  the  con¬ 
sequences  of  failure  are  ser ious , special  restrictions  should  be 
placed  on  the  use  of  the  part  until  the  problem  is  solved.  Once 
a  fix  is  decided,  the  part  should  be  kept  under  surveillance  to 
assure  that  the  "fix"  corrects  the  problem.  Experience  has  shown 
that  all  too  often  a  first  fix  only  corrected  an  obvious  manu¬ 
facturing  defect  leaving  an  underlying  system  design  defect 
still  uncorrected. 

T.4  EXAMPLE  OF  FAILURE  ANALYSIS  REPORT 


.1 

Component  Identification 

(a) 

FAR  Number 

CT-;9-24-146 

Failure  Report  No. 

925944 

(b) 

Part  Name 

ACCELEROMETER 

(c) 

Manu  facturer 

(  XX . 

(d) 

Part  Number 

KA- 1006 

(e) 

Serial 

C07  8B 

(f) 

Next  Assembly 

55-11010 

.2 

History 

(a)  The  accelerometer  reportedly  failed  on  April  19,  1963 
at  the  Astronautics  Standards  Laboratory  during  or  before 


calibration  when  the  lead  wire  to  the  accelerometer  pickup 
was  found  to  be  broken  at  the  pin  connector  junction. 


(b)  The  acceleromci  r  senses  vibration  in  the  booster  jetti¬ 
son  track  support  area.  It  is  one  of  five  accelerometers  used 
on  Centaur  upper-stage  booster. 

(c)  The  accelerometer  consists  of  a  piezoelectric  sensing 
head,  and  a  transistorized  amplifier.  The  sensing  head  is 
connected  to  the  amplifier  by  four  feet  of  cable.  The  ranqe 
of  the  accelerometer  is  +_  30  gravities. 

(d)  Nineteen  of  these  accelerometers  reportedly  failed, 
according  to  A stronautics  trend  reports,  for  various  reasons, 
in  the  six  months  preceding  April  19,  1963.  Four  reportedly 
failed  in  the  last  30  days  of  this  period.  Four  accelerometers 
were  failure  analyzed,  but  none  failed  in  this  mode. 

(e)  Failure  of  the  accelerometer  with  a  broken  lead  wire 
would  make  the  unit  inoperative.  Failure  of  this  mode  during 
flight  would  not  affect  vehicle  operation,  but  would  prevent 
th<=>  monitoring  of  vibration  conditions  during  the  mission. 

.4.3  Analys is 

(a)  Visual  examination  of  the  accelerometer  pickup-confirmed 
the  reported  failure:  the  lead  wire  to  the  sens ing  head  pick¬ 
up  was  severed  at  the  pin  connector  junction.  A  section  througl 
the  pin  connector's  plastic  jacket  revealed  the  characteristics 
of  the  pin  connector  junction  for  a  view  of  the  lead  wire, 

pin  connector ,  plastic  jacket,  and  the  sensing  head. 

(b)  Microscopic  examination  of  the  pin  connector  showed  the 
center  conductor  of  the  coaxial  cable  lead  was  still  soldered 
to  the  center  terminal  of  the  connector.  The  braid  from  the 
coaxial  cable  shield  was  also  still  soldered  to  the  crimping 
ring  of  the  pin  connector  clamping  around  the  cable. 

(c)  Close  examination  of  the  broken  lead  wire  revealed  all 
the  shield  wires  were  twisted  and  bunched  cn  one  side  of  the 
center  conductor  insulation,  and  they  were  broken  off  at 
irregular  lengths-  Tr  is  would  indicate  the  load  wire  had 
been  subjected  to  a  twisting  motion,  resulting  in  ’weakening 
of  the  shi  ’Hmq  cable. 

(d)  The  pin  connector  and  coaxial  cable  adaptor  fitting  were 
externally  coated  with  a  red  sealing  compound,  tending  to  re- 
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strict  the  free  movement  of  the  pin  connector  cap  on  the 
housing.  This  condition  permitted  the  housing  and  the  attached 
coaxial  cable  adapter  to  rotate  while  the  pin  connector  was 
being  secured  to  the  sensmg  head  by  the  pin  connector  cap. 

The  only  wav  to  prevent  the  twisting  of  the  coaxial  cable, 
and  its  cable  adaptor  on  the  pin  connector  during  the  connec¬ 
tion  of  the  pin  connector  to  the  sensing  head,  is  to  grasp 
the  plastic  jacket  surrounding  the  pin  connector  fitting. 

Since  this  adaptor  fitting  has  just  a  3/16- inch  diameter,  and 
1/2-inch  length,  it  must  be  grasped  carefully  if  a  twisting 
cement  is  to  be  prevented  during  connection. 

(e)  The  pin  connector  fitting  on  the  coaxial  lead  cable  does 
not  have  an  adequate  dtsign  strength  at  the  cable  adaptor 
fitting  to  withstand  twisting  or  bending  movement  of  the  cable, 
during  the  connecting  process.  Therefore,  ‘he  shielding  sur¬ 
rounding  the  center  conductor  will  break  when  it  is  subjected 
to  these  extraneous  or  excessive  movements  during  connecting 
or  handling. 

2.4.4  Conclus ions 

(a)  The  accelerometer  failure:  broken  lead  wire,  was  con¬ 
firmed  by  visual  examination.  Microscopic  examination  of  the 
lead  wire  and  the  pin  connector  revealed  the  coaxial  cable 
lead  had  been  subjected  to  a  twisting  motion  that  had  weakened, 
and  then  broke,  the  shield  cable.  The  small  center  conductor 
also  broke,  due  to  fatigue  failure,  as  a  result  of  the  twisting 
movement  to  the  cable. 

(b)  Failure  was  caused  by  a  twisting  action  introduced  into 
the  coaxial  cable  lead  during  connection  of  the  pin  connector 
to  the  sensing  head. 

2.4.5  Recommended  Corrective  Action 

(a)  The  Reliability  Failure  Analysis  Group  recommends  the 
following  corrective  action  to  prevent  breakage  damage  to  the 
accelerometer  coaxial  lead  cable  during  its  connection  into 
the  sensing  head,  and  durirg  handling  before  installation: 

(a)  Use  extreme  care,  when  using  the  coaxial  lead,  not  to 
twist  or  stretch  the  shie’ding  directly  below  the  rubber 
insulation. 


(b)  Grasp  the  plastic  jacket  surrounding  the  pm  connecter 
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fitting  firmly,  when  screwing  the  pin  connector  into  the 
sensing  head. 

(b)  The  Reliability  Failure  Analysis  Group  recommends  the 
following  andor  design  corrective  action  to  prevent  breakage 
damage  to  the  accelerometer's  coaxial  lead  cable,  during 
handling  and  installation  on  Astronautics  missiles: 

(a)  Improve  design  of  the  pin  connector  adaptor  so  the 
coaxial  cable  is  given  adequate  support  and  is  restrained 
from  excessive  twisting  and  bending. 

(b)  Eliminate  use  of  the  red  sealing  compound  on  the 
shank  of  the  pin  connector,  thereby  reducing  the  twisting 
motion  of  the  pin  connector  produced  by  the  interference 
with  the  compound  while  the  connecter  cap  is  threaded  onto 
the  sensing  head. 

2.5  CORRECTIVE  ACTION 


Once  the  cause  of  and  responsibility  for  the  malfunction  has 
been  determined,  positive  steps  must  be  taken  to  assure  that  the 
information  is  used  to  eliminate  the  problem  and  prevent  re¬ 
currence  of  the  malfunctions  (Figure  16-10).  Responsibility 
should  be  assigned  to  an  individual  m  whatever  group  the  action 
to  be  taken  might  lie.  One  useful  technique  is  the  maintenance 
of  a  corrective  action  log  as  suggested  in  Chapter  15.  The 
Corrective  Action  Log  is  a  management  report  listing  all  known 
reliability  (and  other)  problems  with  recommended  solutions  and 
names  of  persons  responsible  for  carrying  out  the  action.  The 
log  is  updated  and  published  weekly  (daily  in  some  critical 
operations).  No  entry  is  removed  until  an  appropriate  action 
taken  by  the  person  responsible  is  accepted  by  the  program 
manager . 

When  a  reliability  problem  of  significance  is  identified  due  to 
a  malfunction  or  part  failure,  the  problem  should  be  logged  and 
assigned  either  to  the  design  group  responsible,  or  to  the 
failure  analysis  group.  The  latter  assignment  is  usually  pre¬ 
ferred,  since  malfunctions  are  commonly  caused  by  defects  in 
manu factur ina  and/or  operator  error.  The  analyst  should  consult 
with  t  e  designer  and  manufacturing  personnel  as  necessary  to 
establish  the  facts.  At  this  time,  the  responsibility  is  trans¬ 
ferred  to  an  "action"  man  in  design  cr  manufacturing,  with 
recommendations  as  to  possible  solutions.  The  "action"  man  is 
not  bound  to  accept  the  recummendat ion  of  the  analyst.  His 
responsibility  is  to  provide  an  acceptable  solution. 


a 
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3  .  TIMING  OF  CORRECTIVE  ACTION 

Up  to  the  present  time,  much  of  the  emphasis  associated  with 
fai lure  reporting  and  corrective  action  has  been  placed  on  pro¬ 
grams  associated  with  production  efforts  and,  to  a  somewhat  less 
extent,  programs  associated  with  field  use  by  the  customer.  This 
is  undoubtedly  brought  about  by  the  fact  that  the  need  for 
corrective  action  is  most  obvious,  perhaps  even  an  absolute 
necessity,  during  this  period.  This  is  also  the  period  during 
which  the  most  data  is  available  on  which  to  base  corrective 
action . 

3.1  ENGINEERING  TESTING 

This  normally  is  the  first  phase  in  the  evolution  of  a  system 
which  produces  satisfactory  test  data.  The  system  design  has 
progressed  from  the  drawing  stage  to  the  model  stage  and  the  design 
itself  should  be  fairly  firm.  Test  data  should  be  reasonably  re¬ 
presentative  of  the  final  system.  This  data  comes  from  testing  at 
all  levels  of  equipment  complexity,  and  is  available  from  a  number 
of  types  of  testing  such  as  environmental  testing,  reliability  life 
testing,  performance  testing,  and  acceptance  test  in  3. 

The  analysis  of  failed  subassemblies  or  systems  is  carried  to  the 
point  of  determining  the  basic  component/part ,  connection,  or 
structural  failure.  This  analysis  is  documented  by  a  Failure 
Analysis  Peport.  This  report  documents  the  failure  data  at  the 
componont/part  level  and  the  corrective  action  required. 

3 . 2  prop7k;tion 

Due  to  the  nature  of  the  production  phase  of  a  program,  there  is 
an  urgent  need  to  obtain  failure  data  and  take  any  required 
corrective  action  as  rapidly  as  possible. 

If  component/part  failures  are  involved,  an  analysis  of  these 
parts  is  normally  required  in  order  to  determine  whether  the 
failure  is  a  result  of  design  or  application,  assemblv,  quality 
control  of  parts  as  received,  or  an  unsatisfactory  type  of  com¬ 
ponent.  The  request  for  this  component  failure  analysis  plus 
the  failure  data  and  the  initiation  of  corrective  action  can  all 
be  documented  by  using  the  Failure  Analysis  Report.  All  of  the 
approaches  mentioned  above  for  failure  reporting  and  corrective 
action  with  regard  to  production  have  dea.'t  primarily  with  single 
failures  or  a  specific  problem  area.  To  determine  any  significant 
trend  in  failures  which  may  require  additional  corrective  action, 
a  summary  report  of  fai3ures  is  needed  in  order  to  determine 
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broad  corrective  action  requirements  and  achievements. 

3.3  OPERATION  RY  CUSTOMER 

This  is  the  final  phase  of  system  evolution  and  is  the  one  toward 
whi^h  all  reliability  efforts  and  corrective  actions  are  directed. 
It  is  the  system  performance  and  reliability  during  this  phase 
which  determines  ^he  value  of  all  prior  reliability  efforts  and 
corrective  action  programs.  Further  correction  action  beyond  that 
taken  in  the  previously  three  phases  can  still  be  taken  and  is 
often  necessary  but,  as  mentioned  earl ’°r,  is  accompanied  by  a 
number  of  severe  penalties. 


4. 


SUMMARY 


An  important  by-product  of  testing  should  be  the  discovery  of 
residual  causes  of  unreliability  and  the  resulting  corrective 
action  to  reduce  or  eliminate  these  causes.  Experience  has  shewn 
that  the  key  to  corrective  action  is  competent  analysis  of  each 
failure . 


4.1  _ PREVIOUS  LACK  OF  INTEREST 

The  need  for  failure  analysis  has  been  hampered  by  the  fact  that, 
traditionally,  test  specifications  have  assumed  that  the  buyer's 
interest  was  limited  to  obtaining  failure  free  devices  that  would 
pass  all  specified  tests  with  a  failure  rate  of  zero.  It  has 
usually  been  stated  or  implied  that  if  failures  occurred,  the 
devices  ceased  being  of  interest  to  the  buyer  and  responsibility 
for  analysis  and  removal  of  the  cause  of  failure  was  the  private 
concern  of  the  contractor.  The  interest  of  the  buyer  would  be 
resumed  after  an  improved  device  had  bet.n  submitted  and  had 
passed  all  tests. 

4.2  RECOMMENDEu  REQUIREMENTS 


This  traditional  treatment  of  failures  occurring  during  test  is 
unacceptable  for  military  equipment.  The  probability  is  high 
that  some  failures  will  occur  during  earlier  testing  programs. 

The  buyer  is  vitally  interested  in  the  diagnosis  of  test-produced 
failures  and  the  Procedure  to  be  followed  must  be  an  inherent 
part  of  the  procurement  specification.  The  following  items  are 
proposed  as  mandatory  specification  requirements. 

(1)  Competent  engineering  failure  diagnosis  is  mandatory  for 

all  failures. 
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^2)  To  the  extent  possible,  each  failure  must  be  assigned  a 
cause  sv.cn  as  test  instrumentation  defect,  test  operator  error, 
part  failure,  part  deterioration,  circuit  tolerance  failure  due 
to  designer's  failure  to  allow  for  normal  part  variations,  etc. 

(3)  Where  failure  occurs  in  an  equipment  under  test,  the 
pertinent  damaging  stresses  must  be  carefully  measured  and 
recorded.  As  an  example,  if  a  capacitor  fails,  the  possible 
damaging  circuit  stress  (voltage,  oi  sometimes  current)  must 
be  measured  and  recorded.  Furthermore,  the  possible  damaging 
external  stresses  (temperature,  humidity,  etc.)  must  also  be 
measured  and  recorded. 

(4)  Where  practicable,  disassembly  and  analysis  must  be  per¬ 
formed  on  failed  or  deteriorated  parts.  A  competent  diagnosis 
must  be  made  in  terms  of  specific  design  features  and  specific 
workmanship,  production  engineering  and  inspection  procedures . 
Where  applicable,  the  failure  diagnosis  shall  include  an  analy¬ 
sis  of  contributing  causes  such  as  inadequate  circuit  design 
(which  will  not,  for  example,  tolerate  normal  part  variations 
plus  expected  part  deteriorations). 

(5)  A  fully  descriptive  report  or  report  section  must  be 
written  for  each  failure.  The  report  must  assign  the  cause 
and  responsibility  and  cover  the  diagnosis  as  outlined  above. 
Where  appropriate,  recommendations  for  corrective  action 
should  alio  be  included. 

4.3  CONTRACTORS  DESIGN  EXPERIENCE 


It  should  be  mentioned  that  often  the  designer's  knowledge  is 
virtually  indispensable  to  adequate  diagnosis.  Thus  the  con¬ 
tractor  should  be  encouraged  to  maintain  a  nucleus  of  his  appli¬ 
cable  design  group  intact  for  the  duration  of  the  reliability 
tests,  and  to  insure  that  this  group  is  available  for  failure 
diagnosis  activity  after  start  of  operational  use.  It  is 
important  that  the  failure  diagnosis  personnel  be  reasonably 
free  from  undue  pressure  by  the  buyer  and/or  other  groups  in  the 
contractor's  organization  that  may  tend  to  restrict  the  investi¬ 
gations  and  produce  inadequate  diagnosis  or  even  concealment  of 
true  problems.  Since  most  of  this  pressure  results  from  efforts 
to  meet  schedule  and  price  commitments  by  the  contractor,  it  may 
be  that  some  relief  must  be  extended  by  the  buyer  in  this  regard 
in  order  to  gain  the  desired  results. 
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Chapter  17 
SPECIFICATIONS 


In  contracting  for  systems  and  equipment ,  the  "specification1  is 
a  primary  tool  used  to  describe  exactly  what  is  required  and  the 
ground  rules  under  which  it  is  to  be  developed.  The  Bureau  is 
interested  in  obtaining  a  product  which  will  perform  specified 
functions  under  well-defined  conditions,  can  be  operated  and 
maintained,  will  withstand  the  rigors  of  handling,  use  and  en¬ 
vironment,  and  can  be  repaired  on  occasion.  To  obtain  this  re¬ 
sult,  the  specification  must  describe  in  precise  detail  what  is 
wanted,  and  frequently  how  the  contractor  must  perform  some  of 
the  work. 

rio  specification  can  be  really  complete  in  itself.  Section  2  of 
standard  specification  format  provides  a  listing  of  referenced 
documents,  frequently  with  the  statement. 

"The  following  specifications,  standards,  drawings  and 
publications,  of  the  issue  in  effect  on  date  of  invitations 
for  bids,  form  a  part  of  this  specification." 

Although  this  statement  contractually  invokes  the  additional  re¬ 
quirements  fully,  administration  of  such  a  requirement  is  not 
automatic  or  invariant  unless  specific  applicability  details  are 
included  in  a  requirement  paragraph  in  the  basic  individual 
system  or  equipment  specification.  This  is  so  because  (a)  con¬ 
tractors  past  experience  is  that  many  of  the  clauses  apparently 
invoked  by  reference  have  neither  been  required  nor  desired  by 
the  customer,  and  (b)  the  conflicts  between  clauses  in  referenced 
documents  often  preclude  a  clear  understanding  by  the  contractor 
or  inspector  as  to  what  is  required. 

The  specification  "tree"  is  a  natural  and  logical  system.  It  is 
difficult  to  conceive  how  the  procurement  of  complex  systems  and 
equipment  could  be  done  without  it.  But  the  weaknesses  must  be 
recognized.  The  sheer  bulk  of  references,  and  their  references 
in  turn  ad  infinitum,  tends  to  obscure  the  meat.  It  is  not  un¬ 
common  for  a  few  basic  specifications  to  in  turn  pick  up  a  total 
of  3000  specifications.  The  referenced  specifications  almost 
invariably  contain  restrictive  clauses,  alternates  and  limita¬ 
tions.  They  nearly  always  contain  material  not  applicable  to 
the  procurement.  Failure  to  define  precisely  what  is  wanted 
ha<?  led  contractors  to  performance  of  work  not  desired  by  the 
customer,  and  conversely,  to  contractor  failure  to  perform 
desired  work. 


\ 
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1.  R&M  SPECIFICATION  CONSIDERATIONS 

Reliability  and  maintainability  are  characteristics  that  cannot 
be  determined  by  inspection.  They  can  never  be  determined  abso¬ 
lutely  in  test  programs.  The  samples  tested  represent  only  a 
small  portion  of  the  systems  built  to  the  design.  So  in  practice, 
the  customer  must  have  some  measure  of  visibility  and  control  of 
the  contractors  effort  toward  reliability  and  maintainability 
quantitative  requirements,  as  well  as  evaluation  of  his  progres¬ 
sive  achievement  of  these  requirements. 

This  visibility  and  control  is  established  through  the  specifica¬ 
tion  imposed  on  the  contractor,  and  it  has  become  customary  to 
include  all  those  tasks,  tests  and  reports  the  contractor  must 
perform  and  submit  for  the  customer  to  have  visibility  and  con¬ 
trol.  At  the  same  time,  there  is  a  growing  feeling  that  there 
is  much  "over-control" ,  and  hope  that  more  incentive  contracting 
will  permit  simpler  controls. 

As  mentioned  above,  there  are  few  instances  where  a  specification 
will  be  applicable,  in  its  entirety,  to  a  particular  program. 

Most  of  the  time  one  or  more  specification  sections  can  be  elim¬ 
inated  because  they  are  not  germane  to  the  program,  or  because 
their  requirements  are  too  severe  or  unrealistic.  Specifications 
are  usually  written  to  cover,  in  a  general  sense,  the  "worst  case" 
application  of  the  equipments  whose  characteristics  they  govern. 

For  example,  the  ambient  temperature  range  may  be  quoted  in  a 
specification  as  being  from  - 54°C  to  65°C.  Equipment  design  to  this 
specification  must  perform  equally  well  at  the  Equator  as  at  the 
North  Pole.  General  purpose  Military  equipment  may  be  required 
to  function  at  either  of  these  locations  3nd  should  be  designed 
to  accommodate  this  wide  latitude  in  ambient  temperature. 

Special  purpose  units  which  will  function  in  a  temperature  con¬ 
trolled  environment,  may  find  the  temperature  extremes  of  such 
a  specification  to  be  far  in  excess  of  what  they  will  ever  ex¬ 
perience.  In  such  cases  the  cognizant  engineer  should  reduce 
the  requirements  of  tv  ^  specification  for  the  procurement  in 
question,  which  will  generally  reduce  acquisition  cost  and  im¬ 
prove  delivery. 

Reliability  and  maintainability  speci f ications  follow  a  similar 
pattern.  Test  programs  specified  for  electronic  systems  are 
normally  based  on  the  fact  that  they  include  large  numbers  of 
identical  parts.  This  fact  controls  the  program  requirements  in 
several  ways:  (a)  demonstration  of  achievement  using  a  sequential 
test  plan  is  usually  practical,  (b)  reliability  to  be  expected  from 
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parts  from  a  particular  supplier  is  relatively  predictable,  and 
(c)  assembly  processes  can  usually  be  controlled  by  statistical 
sampling . 

Where  these  conditions  apply,  the  basic  programs  recommended  by 
the  AGREE  report  (1)  provide  a  reasonable  approach  to  the  achieve 
ment  or  acceptable  reliability.  Where  these  conditions  do  not 
obtain,  different  approaches  are  usually  required. 

To  obtain  a  system  or  equipment  that  meets  Bureau  and  fleet  re¬ 
liability  and  maintainability  needs,  (a)  quantitative  require¬ 
ments  for  reliability  and  maintainability  must  be  specified  and 
(b)  means  for  assuring  that  the  requirements  have  actually  been 
achieved  must  be  established  by  the  specification. 

1.1  R&M  SPECIFICATION  EXPERIENCE 


The  specification  of  reliability  requirements  have  in  the  past 
followed  one  or  more  of  the  following  patterns: 

a)  No  mention  of  reliability  or  quality  control  in  the  specifi- 
cat  ion. 

b)  General  statements,  such  as  "Equipment  shall  have  maximum 
reliability" ,  "The  principle  of  reliability  is  paramount  and  no 
compromise  shall  be  made  with  other  basic  requirements  of  design" 
or  (MIL  E  16400:3.1)  "Reliability  shall  meet  the  needs  of  the 
Naval  Service." 

c)  Requirement  for  inspection  during  manufacture. 

d)  Requirements  for  qualification  prior  to  award. 

e)  General  life  requirement,  with  or  without  specifying  opera¬ 
ting  time  to  first  major  overhaul. 

f)  Specification  of  numerical  Reliability  "goals",  not  contra¬ 
ctual  requirements. 

g)  Specification  of  reliability  requirements  with  verification 
procedure . 

h)  Specifying  that  the  contractor  shall  analyze  the  needs  and 
establish  reliability  requirements  and  verification  criteria. 

i)  Specifying  that  various  program  tasks  (prediction,  testing, 
design  review,  failure  diagnosis)  be  performed. 
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1.2  KINDS  OF  SPECIFICATIONS 

Specifications  designed  to  achieve  tne  required  reliability  and 
maintainability  design  and  development  of  a  system  or  equipment, 
or  to  assure  that  a  procured  equipment,  meets  the  requirements, 
can  be  separated  into  three  groups: 

a)  Establishment  of  quantitative  goals. 

b)  Contracting  for  quantitative  requirements  and  verification. 

c)  Contracting  for  tasks  to  be  performed  by  the  contractor. 

In  the  first  group,  the  quantitative  goal  is  subject  to  other 
requirements  of  the  specification  and  contract.  It  has  no  con¬ 
tractual  standing.  Meeting  the  goal  is  a  matter  of  "good  faith" 
on  the  part  of  the  contractor,  subject  to  compromise  with  con¬ 
tractual  requirements,  to  lack  of  knowledge,  understanding  or 
appreciation. 

Contracting  for  quantitative  reliability  and  maintainability, 
with  verification  by  "demonstration"  or  testing,  provides  the 
Bureau  with  data  to  evaluate  the  equipment.  In  contractual 
terms  it  requires  the  contractor  to  perform  certain  tests  and 
provide  the  data.  The  quantitative  requirement  is  a  performance 
attribute  of  the  specific  system  or  equipment  being  procured. 

It  is  appropriate  to  include  it  in  the  specification,  along  with 
the  use  conditions  and  exact  function  of  the  product.  The  type 
of  test  that  can  be  performed  is  peculiar  to  the  syctem  or 
equipment,  and  should  be  defined  to  some  degree  in  its  specifi¬ 
cation.  The  extent  of  testing  is  dependent  upon  the  requirements 
for  a  particular  application  and  may  vary  from  one  procurement 
to  the  next. 

In  the  third  group,  the  contract  requires  the  performance  of 
specific  tasks.  The  customer  is  buying  efforts  which  are  con- 
sideied  to  achieve  or  at  least  improve  reliability  and  maintain¬ 
ability,  rather  than  any  particular  attribute  of  the  system  or 
equipment.  However,  the  specification  of  analytical  tasks  is 
completely  identifiable  to  the  intended  application  of  the  spec¬ 
ification,  rather  than  a  product  attribute.  As  such  it  should 
not  be  included  in  the  product  specification. 


2 .  SPECIFICATION  LIST 

The  complexity  of  our  specification  system  makes  the.  intelligent 
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selection  of  the  "right"  specification  difficult  and  time-consum¬ 
ing.  To  simplify  the  retrieval  of  speci f icat ion  requirements  the 
following  list  is  provided. 

2.1  RELIABILITY  SPECIFICATIONS  AND  REFERENCES 

These  basic  specifications,  standards  and  references  are  utilized 
in  procurement  of  systems  and  equipment,  to  obtain  required  re¬ 
liability. 

MIL  STD  785  -  Reliability  management  of  Department  of  Defense 
Military  Systems:  This  standard  was  developed  to  provide  industry 
with  guidelines  and  procedures  necessary  for  establishing  and 
implementing  reliability  programs  on  military  systems.  When 
invoked  it  requires  the  contractor  to  establish  and  maintain  an 
effective  and  economical  reliability  assurance  program,  adjusted 
to  suit  the  type  and  phase  of  the  procurement.  See  section  3.2 
below. 

MIL  STD  441  -  Reliability  of  Military  Electronics  Equipment: 

This  standard  was  developed  to  establish  a  procedure  for  the 
development  and  design  of  electronics  equipment  to  insure  re¬ 
quired  inherent  reliability.  It  gives  a  very  general  statement 
of  design  principles  and  considerations  to  be  applied. 

MIL  R  22732  -  Reliability  Requirement  for  Shipboard  and  Ground 

Electronic  Equipment:  This  specification  pi  escribes  the  pro¬ 
cedures  for  management  of  reliability  assurance  programs  in  the 
development  of  shipboard  and  ground  electronic  equipment.  When 
invoked,  it  requires  (a)  knowledge  of  and  application  of  prin¬ 
ciples  of  design  for  '-eliability ;  (b)  establishment  of  r  relia¬ 

bility  assurance  program;  (c)  verification  of  achieved  reliabil¬ 
ity  by  testing  as  specified  in  the  individual  equipment  specifi¬ 
cation,  or  as  proposed  by  the  contractor  if  not  specified;  (d) 
provides  an  alternate  verification  procedure  by  analysis  and 
prediction,  when  approved  by  the  Bureau,  when  demonstrat ion  test¬ 
ing  is  impossible  or  impractical.  See  section  3-2  below. 

M IL  STD  721  -  Provides  general  definitions  of  terms  for  relia¬ 

bility  engineering. 

MIL  STD  756  -  Reliability  Prediction  -  Provides  general  predic¬ 

tion  procedure  based  on  parts  failure  rates,  with  chart  for 
electronic  "active  elements'  . 


MIL  STD  105  -  Provides  sampling  procedures  and  tables  for  in¬ 

spection  by  attributes.  This  standard,  with  Technical  Report 


4 
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#10,  ONR  (Contract  NONR-401 ( 143 ) ) ,  "Factors  and  Procedures  for 
applying  The  MIL  STD  105D  Plans  In  Life  and  Reliability  Inspec¬ 
tion"  may  be  used  to  design  life  and  Reliability  testing  and 
demonstration  plans. 

MIL  HDBK  217  -  Reliability  Stress  and  Failure  Rate  Data. 

MIL  HDBK  H108  -  Sampling  Procedures  for  Reliability  testing. 

MIL  STD  781  -  Demonstration  plans  for  Reliability. 

NAVSHIPS  93820  -  Handbook  for  the  Prediction  of  Shipboard  and 
Shore  Electronic  Equipment  Reliability. 

NAVSHIPS  94501  -  Bureau  of  Ships  Reliability  Design  Handbook. 
MIL  STD  839(USAF)  -  Parts,  with  established  reliability  levels. 
2.2  MAINTAINABILITY  SPECIFICATIONS  AND  REFERENCES 


These  basic  specifications,  standards,  and  refernces  are  utilized 
in  procurement  of  systems  and  equipment  to  obtain  required  main¬ 
tainability  and  availability.  : 

MIL  M  23313  -  This  specification  was  developed  to  prescribe 
>  maintainability  program  requirements  in  the  development  cf  ship¬ 

board  and  shore  electronic  equipment.  When  invoked  it  requires 
the  contractor  to  (a)  establish  a  maintainability  assurance  pro¬ 
gram,  (b)  apply  maintainability  criteria  in  the  design,  and  (c) 
report  and  evaluation  of  the  achieved  maintainability. 

MIL  STD  778  -  Provides  general  definitions  for  maintainability 
engineering. 

MIL  M  19562  -  Provides  instructions  for  the  preparation  of  Main¬ 
tenance  Prints  for  Electronic  Equipment. 

NAVSHIPS  94324  -  Maintainability  Design  Criteria  Handbook  for 
Designers  of  Shipboard  Electronic  Equipment. 

Maintainability  Prediction  Procedure  for  designers  of  Shipboard 
Electronic  equipment  and  systems.  Report  by  Federal  Electric 
Corporation,  Contract  NOBSR  75376. 

2.3  GENERAL  SPECIFICATIONS  AND  REFERENCES 

These  basic  specifications,  standards  and  references,  while  they 
are  not  primarily  concerned  with  reliability  and  maintainability, 
are  utilized  with  significant  impact  on  reliability  and  maintain¬ 
ability: 


v-’ 
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MIL  E  16400  -  Electronic  Equipment,  Naval  Ship  and  Shore,  General 
Specification:  This  specification  (a)  defines  a  basic  design 
philosophy  in  development  of  Naval  Electronics  Equipment  of  util¬ 
izing  the  latest  construction  techniques  with  the  objective  of 
inci easing  reliability,  making  equipment  easier  to  maintain  and 
reducing  overall  cos*-,  (b)  invokes  MIL  R  227 32B  which  requires  a 
reliability  assurance  program,  including  verification  of  relia¬ 
bility,  when  specified  in  the  individual  equipment  specification. 
See  section  3.1  below. 

MIL  STD  202  -  Describes  test  methods,  including  environmental 

and  ovt-rstress  methods  for  the  testing  of  Electronic  and  Electri¬ 
cal  components . 

MIL  STD  210  -  Provides  reference  for  probable  climatic  conditions 

of  the  natural  environment  to  which  Military  Equipment  may  be 
exposed  . 

MIL  STD  242  -  Eleccronic  Equipment  parts  (selected  standards): 

Provides  standard  dimensions,  ratings,  etc.  Selection  of  parts 
from  this  compilation  does  not  constitute  parts  control  in  the 
reliability  sense. 

MIL  STD  446  -  Establishes  uniform  environmental  design  require¬ 

ments  for  use  in  development  and  procurement  of  Electronic  Parts, 
tubes  ana  solid  state  devices. 

Suggestions  for  designers  of  Electronic  Equipment  (Booklet  pre¬ 
pared  by  USNEL) . 

MIL  STD  8b 3  (OSAF)  -  Human  engineering  criteria  for  aircraft, 
missile,  and  space  systems,  ground  support  equipment. 

2.4  SPECIE I CAT  ION  CHART 

Figures  17-9,  17-10  and  17-11  provide  some  perspective  of  the 
variety  of  tasks  covered  by  reliability  and  maintai nabi 1 ity 
specs,  and  the  differences  of  emphasis  across  various  military 
and  NASA  agencies.  Although  chapters  22  and  23  contain  the  re¬ 
commended  spec i f icat ion  language,  the  BuShips  engineer  should  be 
familiar  with  these  other  specifications  that  may  contain  lan¬ 
guage  useful  for  specific  situations. 


R  &  M  SPECIFICATIONS 

RELIABILITY 
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3 .  SPECIFICATION  ABSTRACTS 

3.1  GENERAL  SPECIFICATION  ABSTRACTS  I 

" -  3 

i 

In  the  procurement  of  an  electronic  systems  specification  MIL  E 
16400  (Navy)  General  Specification  for  Electronic  Systems  is 
usually  invoked.  The  speci fication  clauses  relating  to  Reliabil¬ 
ity  are  abstracted  herewith,  showing  the  16400E  section  number  in  j 

parentheses :  I 

(1.1)  Scope .  -  This  specification  covers  the  general  require¬ 
ments  applicable  to  the  design  and  construction  of  electronic 
equipment  and  associated  and  auxiliary  electronic  apparatus  furn¬ 
ished  as  part  of  a  complete  system  intended  for  Naval  ship  or 
shore  applications.  The  intent  of  this  specification  is  to  set 
forth  the  ambient  conditions  within  which  equipment  must  operate 
satisfactorily  and  reliably;  the  general  material,  the  process 
for  selection  and  application  of  parts,  and  to  detail  the  means 
by  which  equipment  as  a  whole  will  be  tested  to  determine  whether 
it  will  so  operate.  Throughout  the  design  and  manufacture  of  the 
equipment,  maximum  effort  shall  be  made  to  attain  the  basic 
design  objectives  in  that  i_he  equipment  will  meet  the  needs  of 
the  Naval  service.  Requirements  applicable  to  individual  equip¬ 
ments  shall  be  as  specified  in  the  individual  equipment  specifi¬ 
cation. 

(1.1.1)  Basic  design  philosophy.  -  The  design  philosophy  of 
Naval  electronic  equipment  is  to  utilize  the  latest  construc¬ 
tion  techniques  with  the  objective  of  increasing  reliability, 
making  the  equipment  easier  to  maintain  and  to  reduce  overall 
cost.  Manufacturers  are  encouraged  to  forward  to  cognizant 
bureaus,  ideas,  proposals  and  suggestions  that  will  result  in 
the  foregoing  objective.  Details  of  this  basic  design  phil¬ 
osophy  are  contained  in  applicable  paragraphs  of  this  speci¬ 
fication.  In  addition,  the  complexity  of  modern  elec  ironies 
systems  and  the  close  relationship  between  the  design  of  the 
equipment  and  the  design  of  the  ship  make  a  closer  liaison 
between  the  shipbuilder  and  the  equipment  manufacturer  de¬ 
sirable.  Even  informal  information  wnich  is  timely,  but  not 
completely  firm  is  often  of  great  mutual  value.  The  Navy 
encourages  the  early  exchange  of  informal  information  between 
shipbuilders  and  equipment  manufacturers. 

(3.1)  Design  Objectives.  -  The  basic  design  objectives  are  that 
the  equipment  will  meet  the'needs  of  the  Naval  service  and  that 
the  final  product  will  reflect  the  utmost  in  simplicity,  have 
maximum  reliability  consistent  with  the  state  of  the  art,  and  be 
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easy  to  install  and  maintain. 

(3.1.2)  Reliability  -  Equipment  reliability  studies  continue 
to  verify  that  the  majority  of  equipment  failures  can  be 
traced  to  the  improper  selection  and  application  of  the  elec¬ 
tronic  parts.  To  assure  that  the  equipment  will  meet  the  re¬ 
quirements  of  Naval  service,  it  is  imperative  that  reliability 
of  operation  be  considered  of  prime  importance  in  the  design 
and  manufacture  of  the  equipment.  The  contractor  shall  employ 
all  methods  possible  in  the  process  of  manufacture  which  will 
assure  quality  and  maximum  reliability  consistent  with  the 
state  of  the  art. 

(3. 1.2. 3)  For  Bureau  of  Ships  equipment,  quantitative  re¬ 
liability  requirements,  in  terms  of  Mean  Time  Between  Fail¬ 
ures  (MTBF) ,  shall  be  that  specified  in  the  individual 
equipment  specification  in  accordance  with  Specification 
MIL  R  22732. 

(3. 1.2. 3.1)  A  Bureau  of  Ships  reliability  assurance 
program,  which  shall  include  the  verification  of  relia¬ 
bility  requirements  shall  be  established  and  maintained 
when  specified  in  the  individual  equipment  specifica¬ 
tion  employing  MIL  R  22732  to  the  extent  applicable. 

(3.1.4)  Ease  of  installation  and  maintenance.  - 

(3. 1.4.1)  Bureau  of  Ships.  -  The  er  ipment  shall  be  de¬ 
signed  so  that  it  can  be  easily  installed  and  maintained. 
Maximum  use  shall  be  made  of  the  design  guides  in  NAVSHIPS 
94324.  Fault  location  accessibility  and  serviceability 
features  which  will  lead  to  simplified  maintenance  shall 
be  a  prime  consideration  in  the  design  (see  3.10.3  and 
3.11.10) . 

(3.1.8)  Failure  reporting.  -  During  research  and  development 
and  service  test  evaluation  of.  electronic  equipment  performed 
by  the  contractor,  prior  to  Government  acceptance,  the  follow¬ 
ing  reporting  is  required: 

(a)  Each  failure  occurrence  in  which  a  part  replacement 
is  involved  shall  be  reported  and  forwarded  to  the 
bureau  or  agency  concerned  using  Bureau  of  Ships  form 
"Electronic  Equipment  Failure/Replacement  Report  DD- 
787  (Proposed)  BuShips  Report  No.  10550-1. 

(b)  An  electronic  equipment 'operating  time  log  report 


shall  be  completed  and  forwarded  to  the  bureau  of 
agency  concerned  each  month  or  upon  completion  of  a 
specified  test,  whichever  is  shorter  using  "Electronic 
Equipment  Operational  Time  Log,  NAVSHIPS  4855"  report¬ 
ing  form." 

(6.4.2)  Individual  equipment  specification.  -  An  individual 
equipment  specification  is  the  detail  specification  covering 
a  particular  equipment. 

(3.4.1)  Selection  of  parts.  -  All  parts  used  in  the  construc¬ 
tion  of  Navy  electronic  equipment  shall  be  in  accordance  with 
the  requirements  specified  herein.  The  selection  of  parts  in 
accordance  with  the  following  order  is  mandatory: 

(a)  MIL  STD  242. 

(b)  Other  Standards,  Specifications  and  requirements 
listed  herein  but  not  included  in  MIL  STD  242. 

(c)  MIL  STD  143. 

All  parts  used  except  as  covered  in  (a)  and  (b)  will  require 
written  approval  in  accordance  with  3. 4. 1.2.  Approval  will 
not  be  granted  for  the  use  of  parts  of  special  or  novel  design, 
except  as  provided  for  in  3.4.5,  where  parts  specified  herein 
are  suitable  and  available.  This  restriction  shall  not  be 
construed  as  restricting  the  use  of  new  or  improved  parts 
which  will  enhance  the  overall  equipment  reliability. 

(3. 4. 1.1)  Standard  parts.  -  Standard  parts  are  those  parts 
specified  ins 

(a)  MIL  STD  242. 

(b)  Other  standards,  specifications  and  requirements 
listed  herein  (where  no  selected  standard  has  been 
established  in  MIL  STD  242  for  that  standard, 
specification  or  requirement). 

(3. 4. 1.2)  Nonstandard  parts .  -  Action  of  the  approval  of 
nonstandard  parts  shall  be  in  accordance  with  MIL  STD  749. 
Tubes,  diodes  and  transistors  are  considered  as  parts  (see 
MIL  STD  749) .  Written  approval  or  disapproval  of  parts 
will  normally  be  taken  within  60  days  after  requires  is 
received  by  the  Bureau  or  agency  concerned. 
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(3.4.2)  Design  Application.  -  Parameters,  such  as  nominal 
ratings,  tolerances,  deratings,  ambient  temperatures,  over¬ 
load  conditions,  and  the  like,  specified  in  individual  part 
specifications  shall  be  applied  when  using  the  parts.  Where 
the  parts  used  are  not  described  by  Military  specifications, 
limits  set  as  a  result  of  Government  Laboratory  tests  estab¬ 
lishing  specific  suitability  for  Naval  service  shall  not  be 
exceeded.  Particular  attention  is  directed  to  the  require¬ 
ments  for  judicious  choice  of  parts  such  that  ratings,  tol¬ 
erances,  and  effects  on  circuit  parameters  after  prolonged 
use  are  carefully  considered.  Specifically,  deterioration 
due  to  permanent  and  substantial  change  of  value (s)  of  a 
given  part  after  aging  is  one  of  the  prime  causes  of  parts 
failure  and  circuit  malfunctioning  (see  6.6).  Attention  is 
also  directed  to  the  necessity  for  considering  the  possible 
degradation  caused  by  temperature  due  to  the  nonjudicious 
choice  of  parts  location.  It  is  obviously  undesirable  to 
place  many  parts  in  the  vicinity  of  items  generating  substan¬ 
tial  heat,  such  as  transformers  and  tubes.  Attention  should 
be  directed  to  the  cooling  of  cylindr ically  shaped  parts  by 
use  of  "chimmey  effect"  by  mounting  in  a  vertical  plane. 

Where  parts  are  stacked,  the  heat  gradient  and  subsequent 
"hot  spot"  of  the  uppor  stack (s)  should  be  given  careful 
cons ideration. 

(3.6.6)  Fasteners  and  assembly  screws.  -  All  external  fasten¬ 
ers  and  assembly  screws  which  are  manipulated,  loosened,  or 
removed  in  normal  process.es  of  installation  and  maintenance 
of  equipment  shall  be  of  such  as  to  provide  strong  contrast 
with  the  color  of  the  surface  upon  which  they  appear.  Other 
external  fasteners  and  assembly  screws  shall  be  of  the  same 
color  as  the  surface  upon  which  they  appear.  Metallic  cou¬ 
ples  which  will  cause  galvanic  corrosion  shall  not  be  employed 
to  obtain  contrasting  color  for  Bureau  of  Ships  contracts  only. 

(3.9.2)  Preferred  circuits.  -  In  the  interests  of  standard¬ 
ization  of  circuits,  use  of  the  standard  preferred  parts,  and 
ultimately,  the  collection  of  circuit  performance  reliability 
data,  circuits  shall  be  selected  from  Standard  MIL  STD  439, 
where  applicable. 

(3.11.10)  Accessibility.  -  The  arrangement  of  parts  shall  be 
such  that  replacement  or  adjustment  of  any  part  is  possible 
without  removal  of  or  damage  to  adjacent  parts.  All  parts 
shall  be  readily  accessible  for  replacement  or  repair. 


(3.13.4)  Arrangement .  -  Controls  shall  be  so  arranged  as  to 
facilitate  smooth  and  rapid  manipulation.  Indicators  shall 
be  designed  and  arranged  to  insure  readability  under  service 

conditions.  Locations  of  similar  controls  and  indicators  on 
different  panels  shall  correspond  insofar  as  practicable. 

(3.13.13)  Time  meters.  -  Time  meters  shall  be  provided  for 
electronic  equipment  to  indicate  elapsed  time  for  both  stand¬ 
by  and  operation.  The  circuits  to  be  monitored  shall  be  as 
specified  in  the  individual  equipment  specification. 

(3.14.4)  Parts  identification  by  reference  designations 
(symbol  designations).  -  In  order  to  facilitate  maintenance, 
each  part  assembled  in  a  major  unit  and  set  shall  be  identi¬ 
fied  by  an  appropriate  reference  designation  in  accordance 
with  Standard  MIL  STD  16. 

(4.1.1)  Quality  control  system.  -  The  contractor  shall  pro¬ 
vide  and  maintain  a  quality  control  system  acceptable  to  the 
Government  for  the  supplies  covered  by  the  contract.  The 
system  of  quality  control  shall  be  in  accordance  with  MIL  Q 
9858.  The  procedures  outlined  in  MIL  Q  9858  shall  serve  to 
supplement  and  implement  the  design,  performance  and  test 
requirements  of  the  individual  equipment  specification. 

(4.3)  Preproduction  inspection.  -  Preproduction  inspection  shall 
consist  of  all  examination  and  testing  necessary  to  determine 
compliance  with  the  requirements  of  the  individual  equipment 
specification  and  unless  otherwise  specified  therein  shall  in¬ 
clude  the  examination  and  tests  specified  hereinafter.  Where 
preproduct  i^r.  inspection  has  been  made  on  an  earlier  model,  a 
careful  check  shall  be  made  to  determine  that  all  corrective 
measures  found  necessary  as  a  result  of  such  inspection  have 
been  carried  out.  (For  each  of  the  following,  the  applicable 
spec  section  number  is  given). 


Surface  examination  Power 

Operating  test  Radio  interference  &  radiation 

Weights  and  dimensions  Frequency  spectrum  signature 

Supply  line  voltage  &  frequency  Controls  and  control  circuits 
Water  cooling  Accelerated  life  test 

Heat  test  Shock,  vibration,  &  inclination 

Enclosure  test  Salt  spray  test 

Temperature  Reliability 

Humidity 
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(4.4.3)  'roduction  control  inspection.  -  Production  control 
inspection  shall  be  conducted  on  a  sampling  basis  and  shall 
encompass  functional  and  performance  tests  throughout  the 
required  range;  tests  which  will  detect  any  deterioration  of 
the  design  by  wear  of  such  items  as  dies,  molds,  and  jogs, 
and  by  substitution  of  different  parts?  tests  which  detect 
deviations  in  the  processing  of  the  materials;  tests  to 
determine  temperature  rise  produced  in  operation  and  ability 
of  equipment  to  withstand  this  heat;  tests  of  efficiency;  and 
tests  of  the  performance  with  other  equipment  in  a  system. 
These  tests  shall  be  performed  on  the  complete  equipment  as 
offered  for  delivery.  Unless  otherwise  specified  in  the  in¬ 
dividual  equipment  specification,  production  control,  inspec¬ 
tion  shall  include  the  following  (again  section  numbers  are 
given)  ; 


Weights  and  dimensions 

Supply  line  voltage  &  frequency 

Water  cooling 

Heat  test 

Enclosure 

Power 


Radio  interference  & 
radiation 

Equipment  freq.  spectrum 
s ignature 

Controls  &  control  circuits 
Weld  Test  (when  required) 
Reliability 


(4.4.4)  Environmental  tests.  -  Environmental  tests  shall  be 
conducted  to  prove  the  durability  of  the  materials,  parts, 
major  units,  and  the  equipment  as  a  whole;  life  tests;  simu¬ 
lated  service  tests;  tests  of  the  effects  of  changes  of  en¬ 
vironment  (such  as  extremes  of  temperature  and  humidity, 
effect  of  salt  air);  and  tests  of  the  effects  of  shock, 
vibration,  inclination,  and  hard  usage.  unless  otherwise 
specified  in  the  individual  equipment  specification,  these 
tests  shall  include  the  following  (again  section  numbers  are 
given)  : 


Temperature 

Humidity 

Accelerated  life  test 

Shock,  vibration  and  inclination 

Salt  spray  test 


(4.5.13)  Accelerated  life  tests.  -  If  the  equipment  is  de¬ 
signed  for  water  cooling,  these  tests  shall  be  conducted  with 
water  at  35°  t  2°C.  inlet  temperature  continuously  circulating 
through  the  water  circuit.  The  equipment  shall  be  subjected 
to  the  following  conditioning  and  tests. 


(4.5.13.2.2)  The  test  cycle  specified  in  4.5.13.2.1  shall 
be  repeated  without  interruption  for  a  period  of  360  hours 
(15  complete  days). 

(4.5.14.2.5)  Test  data.  -  Test  data  accumulated  during 
the  accelerated  life  tests,  including  details  of  all  fail¬ 
ures,  shall  be  provided  and  shall  be  included  in  the  re¬ 
port  on  preproduction  inspection. 

(4.5.17)  Reliability.  -  Verification  of  reliability  require¬ 
ments  shall  be  performed  as  specified  in  the  individual 
equipment  specification  employing  Specification  MIL  R  22732 
to  the  extent  applicable. 

(6.2)  Since  this  specification  is  general  in  scope  and  covers 
only  the  construction  practices  and  the  conditions  under  which 
equipment  for  Naval  ship  or  shore  use  must  operate,  the  details 
of  performance  of  the  equipment  under  the  conditions  stated  herein 
and  the  ordering  information  must  be  specified  elsewhere.  Atten¬ 
tion  of  design  engineers  is  invited  to  the  items  listed  below 
which  should  be  covered  in  the  individual  equipment  specifications. 

a)  Detail  performance  requirements  for  the  particular  equipment. 

b)  Class  of  equipment  (see  1.2  and  3.8.1). 

ff)  Modification  of  production  control  inspection  (see  4.4.3). 

gg)  Modification  to  environmental  tests  (see  4.4.4). 

kk)  Nominal  conditions  for  accelerated  life  test  (see  4.5.14.1.1). 

c)  Mean  time  between  failures  (see  3. 1.2. 3). 

(6.5)  This  specification  should  be  referenced  in  all  individual 
equipment  specifications,  including  speci f ications  for  equipment 
in  tne  development  stage,  in  order  to  insure  the  use  of  stan¬ 
dard  parts  rather  than  nonstandard  or  special  parts. 

(6.6)  The  high  reliability  requirement  becomes  a  critical  pro¬ 
blem  with  increasing  complexity  of  equipment  in  terms  of  greater 
number  of  parts.  Even  part  failure  rates  in  the  very  low  percent 
level,  when  multiplied  by  a  large  number  of  parts,  present  a  high 
probability  of  equipment  failure.  Increasing  availability  of 
miniaturization  techniques,  of  new  high-reliability  parts  and 

new  assembly  methods  will  lead  to  space  and  weight  savings.  In  each 
case,  consideration  shall  be  given  to  the  use  of  weight  and  space 
savings  for  ultra-conservative  design  in  critical  circuits  or  the 
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use  of  redundant  circuitry  in  these  areas.  Redundancy  techniques 
shall  be  considered  only  when  it  becomes  evident  from  analysis  of 
the  tentative  design  that  the  high  reliability  requirements  cannot 
be  met  in  any  manner. 

(6.9)  The  NEL  Reliability  Design  Handbook  is  eiteu  as  a  design 
guide.  Though  very  useful  as  a  design  guide,  the  applicable 
specifications  under  the  specific  contract  shall  govern  in  all 
cases  of  discrepancy,  deviation  or  conflict  between  these  con¬ 
tract  specifications  and  this  design  guide  document.  All  devia¬ 
tions  from  specification  requirements  under  a  contract  shall  be 
approved  by  the  Bureau  or  agency  concerned. 

(6.9.2)  An  additional  guide  for  designers  is  the  booklet 
titled  "Suggestions  for  Designers  of  Electronic  Equipment" . 
This  booklet  also  references  MN-8681B,  "Vibration  Problems  in 
the  Design  of  Shipboard  Electronic  Equipment" .  This  is  a 
35mm  sound  movie,  16  minutes  in  length,  which  may  be  borrowed 
from  any  Navy  district  film  library,  Bureau  of  Ships  and  U.  S. 
Navy  Electronics  Laboratory  (USNEL) ,  San  Diego  52,  California. 
The  booklet  may  be  obtained  free  of  cost  from  USNEL. 

3.2  RELIABILITY  SPECIFICATION  ABSTRACTS 

Specification  sections  of  MIL  STD  785  and  MIL  R  22732B  relating 
to  a  common  objective  are  grouped  together  in  this  section,  for 
comparison  and  used  in  procurement.  Again,  the  specification 
section  numbers  are  in  parentheses: 


Note  that  there  is  much  duplication  of  intent,  using  differei.r 
words  for  the  same  or  closely  related  objectives.  For  example, 
quantitative  requirements  are  called  out  by  3.2.1  (3.2.1)  below, 
as  well  as  3.1  (3. 1.2.3)  above.  Parts  control  is  specified  in 
3.2.6  below,  also  in  3.1  (3.4.1  above).  Verification  is  covered 
in  3.2,2  below  and  in  3.1  (4.4.4)  and  (4.5.18)  above.  However 
these  are  quite  gross  relationships.  It  has  been  found  imprac¬ 
tical  to  actually  compare  paragraph  details.  Chapters  22  and  23 
do  extract  all  parts  of  MIL  STD  785  by  actual  contractor  task, 
which  sometimes  involves  extraction  of  a  single  sentence  from  a 
wide-ranging  paragraph. 


i 

3.2.1  Quantitative  Requirements 


MIL  STD  1 8 5  (3.2.1)  Quantitative  Requirements.  The  system  re¬ 
liability  objectives  and  minimum  acceptable  requirements  shall 
be  as  specified  contractually.  The  minimum  acceptable  reliabil¬ 
ity  requirements  for  some  major  subsystems  and  equipments  may  be 
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included  in  appropriate  sections  of  the  system  specification. 

The  values  not  established  by  the  procuring  activity  shall  be 
established  by  the  system  contractor  at  a  contractually  speci¬ 
fied  control  point  prior  to  release  of  design  for  initial  fabri¬ 
cation  of  specified  articles. 

MIL  R  22732B  (2.1)  Lctoil  requirements  for  individual  equipments 
Detail  reliability  requirements  or  exceptions  applicable  to 
particular  equipments  shall  be  specified  in  the  individual 
equipment  speci f ication .  In  the  event  of  any  conflict  between 
requirements  of  this  specification  and  the  individual  equipment 
specification,  the  latter  shall  govern. 

3.2.2  Reliability  Verification:  MIL  STD  785  requires  the  con¬ 
tractor  to  develop  a  plan  for  demonstrating  achieved  reliability 
at  specified  milestones.  MIL  R  22732  gives  detailed  test  re¬ 
quirements  (based  on  AGREE  plan)  that  tests  the  required  relia¬ 
bility  against  an  alternate  hypothesis  of  2/3  the  required  value 
with  consumers  and  producers  risks  of  10%,  (see  chapter  11). 

This  is  not  equivalent  to  demonstration  of  the  required  reliabil¬ 
ity.  It  actually  demonstrates  that  the  reliability  is  at  least 
2/3  of  that  required  with  90%  confidence.  In  the  event  the 
Bursau  decides  to  u  e  some  test  plan  other  than  the  AGREE  pla*. 
and  particularly  if  a  new  one  is  prepared,  it  would  be  well  to 
ha«e  the  plan  checked  by  a  statistician. 

MIL  STD  78  5  (3.5.  11  Test  R^gui ro-ents  for  Development  C-uaiifica- 
tion  and  Acceptance:  A  planned  and  scheduled  program  of  func¬ 
tional  and  environmental  testing  of  equipment  shall  be  conducted 
during  design  and  development  phases  to  estimate  achieved  relia¬ 
bility  and  to  provide  feedback  of  data  as  a  basis  for  making  re¬ 
liability  improvements.  The  development  testing  program  snail 
confirm  adequacy  of  selection  of  comi  orients  and  parts,  Determine 
capabilities  and  safety  margins,  evaluate  drifts  of  component 
parameters  with  time,  and  determine  f «i lure-modes  and  relative 
failure-rates.  If  such  data  are  not  available,  all  items  of  the 
system  determined  by  the  reliability  studies  (3.2.2  and  3.3.3) 
to  have  a  significant  beari.no  on  inherent  reliability  shall  be 
tested  early  in  the  development  program,  unless  other  valid  proof 
of  adequacy  can  be  presented. 


-T- 
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(3.5. 1.1)  Environmental  Requirements  '•  r  1.  ;u  i  p~  <_ r.  f  r  ts  i  jn  an  ; 
testing :  If  maximum  environmental  stress  ns  ne  t 

been  established  by  the  procuring  act  iv  1 1  /  thos-  -shall  fc< 
estimated  from  experience  on  past  pr<>  irams ,  an:  a  t  st  ir;  - 
gram  for  development,  qua  1  i  f  i  cat  i  on ,  an  i  accept  %lr.r>  sh  »  1 1 
generated  on  this  basis.  Development  and  qualification  tv  ats 
shall  be  planned  to  evaluate  the  adequacy  of  design  >  i  eq> 

ment  for  the  expected  conditions  in  the  ..  o  ’-'.’nm-nt  ( . 

ground  operation,  launch,  flight  and  orbit).  Tin:  t^st  plans 
shall  include  consideration  of  eg  pnv-nt  ,  1  cat;  p. ,  insult  t  ion 

shock-mount  ing  ,  truss  mounting  etc.  Env  i  r  noun’  >  1  probl<  m 
areas  shall  be  identified  at  the  system,  sabs /stem,  comporent 
and  part  level,  and  the  effects  of  these  problems  o>"  system 
reliability  shall  be  stimated  on  equipments,  components,  or 
parts  identified  as  critical.  Detailed  and  specific  review 
of  environmental  factors  affecting  reliability  shall  nv  per¬ 
formed.  In  addition  to  quali f icat ion  and  acceptance  testing, 
additional  testing  shall  be  performed  on  critical  items,  uch 
as  life  testing  or  failure-medo  testing  t>.  assess  the  af’ects 
of  the  environments  on  such  critical  items,  ami  t  ieterm; ne 
adequacy  of  safety  margins  incorporated  by  svste"  design, 
subject  to  approval  by  the  procuring  activity. 


des  ign , 


(3. 5. 1.2)  Component  Part  Testing:  All  component  parts 
used  in  production  equipment  shall  be  ass i ;nei  a  reliabi 
index,  failure-rate,  or  expected  probabi 1 i ty  o*  failure 
stated  stress  levels.  The  reliabi !  :  ty  test  trace. hires  •. 
applicable  military  part  sped  fi--.it  ions  and  test  in:  siw 
cations  shall  be  used.  Where  the  contractor  deems  thes», 
procedures  not  applicable,  he  shall  submit  a  just,  t  f  teat  i 
non-appl icabi  1  ity  and  a  desor  ipt  ion  of  t he  test  pc-’cedur 
which  he  plans  to  use.  A  current  recot  i  >f  the  results 
be  maintained.  The  test  data  shall  be  r  'tanvd  ♦•or  a  mi 
period  of  2  years  from  complct  ion  of  --'ontract.  The  test 
shall  be  made  available  to  intormation  and  lata  exchang* 
activities  upon  request  of  t  no  procurin'-  activity. 


t es t  m  :  spec l f l- 
r  deems  these  ♦  est 
a  jus  till  at  i  ot  o  f 
test  procedure* 
the  r  t.-su  Its  s!  a  1 1 
i ned  ♦•or  a  min -mum 
ct. .  The  test  oat  a 


(3 .5. 1. 3)  Maximum  Pre- a c c opt once  Op  a  t  at  ion :  Th  o  c  on t rate  r 
shall  provide  and  maintain  a  current  list  of  it«*ms  having 
critically  limited  useful  lives  (total  operating  time  or  oper¬ 
ating  cycle)  in  their  application.  Deri vat i  ->n  .  f  maximum 
allowable  operating  time  (or  cycles  of  operation)  shall  be 
clearly  defined  with  elements  of  data  *»nd  methods  of  ccmputa- 
tions.  The  contractor  shall  propose  for  approval  the  time  or 
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number  of  equivalent  operating  cycles  that  not  to  be  ex¬ 
ceeded  prior  to  acceptance  of  the  contractor’s  product.  He 
shall  ensure  that  each  such  item  has  its  total  operating  time 
or  number  of  equivalent  operating  cycles  recorded,  starting 
with  and  including  its  initial  functional  test,  whether  at  the 
contractor's  facility  or  a  supplier's  facility.  Upon  mutual 
agreement  between  the  procuring  activity  and  the  contractor, 
any  item  may  be  dropped  from  the  above  list,  or  its  limit 
revised,  when  changes  in  the  items  useful  life  indicate  the 
need  for  such  revisions. 

(3.5.16)  Reliability  Demonstration 

(3.5.16.1)  Initial  Plan;  An  initial  plan  for  demonstr ation 
of  achieved  reliability  at  specified  milestones,  includin'^ 
estimated  number  of  test  articles  and  if  not  specified  by  the 
procuring  activity  a  quantitative  estimate  of  the  confidence 
level,  shall  be  prepared  by  the  contractor  and  submitted  in  a 
section  of  the  reliability  program  plan.  The  general  plans 
for  demonstration  of  reliability  shall  include  trade-off 
curves  showing  number  of  test  articles  and  operating  test 
tj.me  or  test  effort  versus  confidence,  and  will  „ncompass 
testing  at  the  system  "ajor  element  level,  and  major  sub¬ 
system  or  component  vels  separately  and  in  combination. 

(3.5.16.2)  Final  Plan;  Final  plan  for  demonstrating  achieved 
reliability  shall  include  any  revisions  to  data  in  the  initial 
plan,  and  the  ground  rules  enu  conditions  for  deciding  whether 
a  test  shall  be  classified  as  a  success  or  failure,  or  shall 
be  excluded  due  to  invalid  test  data.  Reliability  demonstra¬ 
tion  plans  shall  apply  all  results  of  testing  and  operations 
from  which  valid  reliability  measurement  or  assessment  can  be 
obtained.  Engineering  tests  and  analyses,  e.g.,  test  to  fail¬ 
ure  concept  -,  shall  be  included  to  supplement,  statistical 
measures.  The  milestones  that  are  to  constitute  demonstra¬ 
tion  of  contract  compliance  shall  be  established  and  incor¬ 
porated  in  the  contractual  documents.  Specific  plans  for 
conducting  a  reliability  demonstration  shall  be  submitted  for 
approval  at  the  time  specified  by  the  procuring  activity. 

(3.5.16.3)  Test  Plans  :  The  test  plans  contained  in  MIL  STD 
781,  when  applicable,  shall  be  applied. 


4 
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MIL  R  227  32B  (3.2.5)  Prototype  (pre-production)  models  :  When 
the  procurement  includes  the  fabrication  of  prototype  (pre-pro¬ 
duction)  models  of  the  equipment,  the  contractor  shall  perform 
a  reliability  demonstration  test  to  assure  that  the  reliability 
required  in  the  individual  equipment  specification  is  character¬ 
istic  of  the  equipment  desi  gn. 

(3. 2. 5.1)  Reliability  Deno.-u  t r "tion :  Reliability  damonstra- 
tion  tests  shall  be  per  formed  in  accordance  with  4.2  and  4.3. 
Tested  equipments  shall  exhibit  a  mean-time-between- failures 
(MTBF)  equal  to  or  greater  tv  ,n  that  specified  in  the  indiv¬ 
idual  equipment  speci fication  as  determined  by  4.2.6.  No 
decision  to  accept  or  reject  shall  be  made  until  each  equip¬ 
ment  tested  has  accumulated  an  operating  time  of  at  least 

3/2  times  the  specified  MTBF  without  specific  approval  by 
the  Bureau  or  agency  concerned.  If  the  test  terminates  in  a 
reject  decision,  the  contractor  shall  indentify  the  cause  or 
causes  of  such  a  decision  from  an  analysis  of  the  failure 
data  accumulated  during  the  test  and  propose  corrective  action 
necessary  to  eliminate  the  causes  of  unreliability  identified. 
When  it  is  impossible  or  impractical  to  require  reliability 
*  demonstration  and  testing  in  accordance  with  4.2  and,  upon 

'  specific  approval  by  the  bureau  or  agency  concerned,  the 

reliability  assurance  procedure  of  paragraph  4.3  shall  be 
applied. 

(3. 2. 5. 2)  Reporting :  The  results  of  the  reliability  demon¬ 
stration  test  shall  be  summarized  in  a  report  to  the  procure¬ 
ment  agency.  This  report  shall  contain  the  records  specified 
in  4.2.5  and  an  analysis  of  the  information  they  contain. 

(3.2.6)  Production ;  When  equipments  are  committed  to  production, 
the  contractor  shall  perform  reliability  production  tests  on  pro¬ 
duction  units  to  demonstrate  that  the  level  of  reliability  re¬ 
quired  in  the  individual,  equipment  specification  is  maintained 
during  the  production  process. 

(3. 2. 6.1)  Reliability  Production  Tests;  Reliability  produc¬ 
tion  tests  shall  be  performed  on  samples  tahen  from  each 
periodic  production  lot  in  accordance  with  the  criteria  of 
4.2.  Unless  otherwise  specified  in  the  individual  equipment 
specification,  the  periodicity  for  reliability  production 
testing  shall  be  one  month.  Tested  equipments  shall  exhibit 
a  MTBF  equal  to  or  greater  than  that  specified  in  the  indiv- 
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ideal  equipment  specification  as  determined  by  the  criteria 
of  4.2.6.  No  untested  production  units  shall  be  released  as 
acceptable  for  shipment  until  the  reliability  test  for  that 
production  lot  results  in  an  accept  decision  without  specific 
approval  by  the  Bureau  or  agency  concerned. 

(3. 2. 6. 2)  Repo. ting :  The  results  of  each  reliability  pro¬ 
duction  test  shall  be  summarized  in  a  report  to  the  procure¬ 
ment  agency.  This  report  shall  contain  the  records  specified 
in  4.2,5  .nd  an  analysis  of  the  information  they  contain. 

(4.2)  Reliability  Assuranc e  by  Testing  : 

(4.2.1)  Reliability  Tests:  Reliability'  tests  shall  be  con- 
du  :ted  on  samples  of  the  prototype  and  production  units  of 
equipments  that  have  minimum  or  specified  MTBF  requirements . 

If  a  specific  reliability  test  plan  is  not  indicated  in  the 
individual  equipment  specification,  then  60  days  prior  to 
testing,  the  contractor  shall  submit  for  approval  a  detailed 
reliability  test  plan  that  incorporates  at  least  the  features 
specified  by  this  docur  nnt  and  by  the  individual  equipment 
specification.  Task  Group  Reports  2  and  3  of  Reliability  of 
Mi  l it ary  Electronic  Equipment  may  be  used  as  a  guide  for  com¬ 
pleting  the  detailed  test  plan.  Plans  for  reliability  tests 
integrated  with  other  quality  conformance  inspection  tests 
may  bv  submitted  for  approval  to  the  bureau  or  agency  con¬ 
cerned  . 

(4.2. 1.1)  Test  Details:  The  contractor  and  the  procuring 
group  shall  reach  a  written  agreement  specifying  all  as¬ 
pects  of  the  reliability  tests,  including  reporting,  forms 
before  starting  the  tests.  Rules  for  scoring  failures  shall 
be  exact.  The  performance  characteristics  to  be  measured 
and  their  tolerances  shall  be  covered  in  the  individual 
equipment  specification.  They  shall  be  kept  to  a  minimum 
compatible  with  determination  of  satisfactory  and  unsatis¬ 
factory  performance.  The  environment  in  which  the  equip¬ 
ment  is  tested,  any  preventive  maintenance  to  be  permitted, 
and  other  details  of  the  test  program  shall  all  be  submitted 
to  the  procuring  activity  and  approved  before  the  tests 
begin.  When  approved  by  the  procuring  activity,  the  con¬ 
tractor  may  elect  to  include  any  or  all  quality  conform¬ 
ance  inspection  tests  specified  into  the  individual  equip¬ 
ment  specification  or  MIL  E  16400  as  part  of  uhe  reliabil¬ 
ity  test  with  no  change  in  accept-re ject  criteria. 
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(4.2.2)  Sample  Size:  The  number  of  samples  to  be  tested  will 
be  specified  in  the  individual  equipment  specification.  When 
not  specified,  the  contractor  shall  propose  a  sampling  plan 
for  approval  by  the  procuring  activity. 

(4.2.3)  Environment :  The  following  test  levels  shall  be 
used  for  determining  the  environment  to  be  imposed  during  re¬ 
liability  testing;  the  selection  of  the  particular  guide 
shall  be  specified  in  the  individual  equipment  specification. 

(4.2.5)  Recorded  Data;  From  the  start  to  the  conclusion  of 
the  test,  the  contractor  shall  maintain  a  continuous  adequate 
and  accurate  record  of  measurements  of  performance,  test  time, 
test  operator's  observation,  failures,  and  test  facility  con¬ 
ditions.  The  data  taken  during  the  test  shall  be  the  least 
necessary  to  complete  the  following:  (a)  Operational  sheet; 

(b)  Log  of  equipment  failures  and  operating  time;  (c)  Failure 
report;  (d)  Equipment  logs. 

(4. 2. 5.1)  Operation  Sheet:  The  operation  sheet  shall  be 
designed  to  provide  a  continuous  record  of  the  test  sample 
and  test  facility  performance. 

(4. 2. 5. 2)  Log  of  equipment  failures  and  operating  time: 

The  log  of  equipment  failures  and  operating  time  shall 
contain  the  information  necessar »  for  an  accept  or  reject 
decision.  The  heading  of  the  log  shall  identify  the  test, 
the  specific  equipments  under  test,  and  the  person  respon¬ 
sible  for  the  log.  The  body  shall  contain  the  following 
information:  (a)  Entry  number;  (b)  Date  and  time  of  entry; 

(c)  Identification  of  equipment  that  failed;  (d)  Accumu¬ 
lated  operating  time  of  all  equipments;  (f)  Normalized 
test  time  (item  (e)  divided  by  specified  MTBF) ;  (g)  Total 
number  of  failures  observed  for  all  equipments  on  test. 

An  entry  shall  be  made  at  the  occurence  of  each  apparent 
equipment  failure.  If  failure  diagnosis  reveals  that  ths 
test  speciment  was  not  at  fault,  the  failure  may  be  deleted 
upon  appropriate  reference  to  the  operation  sheet.  Upon 
accumulation  of  enough  time  or  failures  for  either  an 
accept  or  reject,  the  test  shall  be  concluded  with  an 
appropriate  entry. 


(4. 2. 5. 3)  Failure  Report :  Completion  of  a  failure  report 
to  sufficiently  describe  all  pertinent  circumstances  atten¬ 
dant  to  each  equipment  failure  shall  be  mandatory.  The 
failure  report  shall  have  two  main  parts;  one  shall  report 
the  exact  nature  of  the  failure,  and  the  other  shall  re¬ 
port  the  cause  to  the  fullest  extent  possible.  The  first 
part  shall  describe  the  symptoms  and  the  diagnosis  action 
taken,  how  the  equipment  was  repaired,  identification  of 
parts  replaced  or  adjustments  made,  and  what  the  effect  of 
the  repair  was.  The  second  part  shall  include  an  analysis 
of  the  failed  part,  an  analysis  of  the  circuit,  and  pro¬ 
posals  for  action  to  prevent  recurrence  of  the  failure. 

(4. 2. 5. 4)  Equipment  Log:  There  shall  be  an  equipment  log 
for  each  unit  tested.  It  shall  remain  attached  to  the  unit 
throughout  the  test  to  provide  a  complete  history  of  the 
equipment.  The  equipment  log  shall  report  the  performance 
of  the  equipment,  any  adjustments  or  repairs,  and  the  oper¬ 
ating  time  accumulated  during  the  reliability  test. 

3.2.3  Planning  Tasks;  MIL  STD  785  (paragraph  3.1)  provides  a 
clear  statement  of  requirements  for  the  contractor  to  conduct  his 
work  on  the  contract  in  a  logical  orderly  manner.  It  is  parti¬ 
cularly  applicable  to  CPFF  contracts,  when  it  can  be  administered, 
where  it  can  be  a  very  useful  tool  in  obtaining  visibility  of  the 
contractor's  performance.  MIL  R  22732,  while  requiring  that  the 
contractor  have  a  program,  does  not  require  its  documentation  and 
hence  limits  visibility  and  control  over  adequacy  of  planning. 

MIL  STD  785  (3.1)  Reliability  Assurance  Program;  The  contractor 
shall  establish  and  maintain  an  effective  and  economical  relia¬ 
bility  assurance  program,  planned,  integrated,  and  developed  in 
conjunction  with  other  planning  functions.  The  program  shall  be 
adjusted  to  suit  the  type  and  phase  (design,  development,  pro¬ 
duction)  of  the  procurement.  The  program  shall  be  based  upon 
the  severity  of  the  requirements,  the  complexity  of  the  design, 
the  quantity  under  procurement,  and  the  manufacturing  techniques 
required.  The  program  shall  assure  adequate  reliability  consider¬ 
ation  throughout  all  aspects  of  the  design,  development,  or  pro¬ 
duction  as  necessary  to  meet  the  contractual  reliability  require¬ 
ments  . 
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(3.3.1)  Proposed  Reliability  Program  Plan;  The  contractor's 
proposed  reliability  program  plan,  in  accordance  with  the 
requirements  of  the  work  statement  and  this  standard,  shall 
be  submitted  as  a  separate  and  complete  entity  within  the 
contractor’s  proposal  for  the  system.  The  proposed  plan  must 
be  an  integrated  effort  within  the  total  program  plan;  it 
shall  provide  specific  information  as  to  how  the  contractor 
will  meet  specified  quantitative  reliability  requirements 
during  development  and  manufacture  including  the  design 
concepts  to  be  utilized.  The  proper  manner  of  demonstra¬ 
ting  reliability  at  stated  confidence  levels  shall  be  des¬ 
cribed.  The  proposed  reliability  program  plan,  as  approved 
by  the  procuring  activity  will  become  a  contract  compliance 
document;  reliability  tost  plans  must  be  an  integral  part 

of  the  program  test  plan. 

(3.3.2)  Reliability  Organization:  The  program  plan  shall 
(1)  identify  the  organization  and  the  personnel  responsible 
for  managing  the  overall  reliability  program,  and  (2)  shall 
clearly  define  its  responsibilities  and  functions  including 
both  policy  and  action.  It  shall  stipulate  the  authority 
delegated  to  this  organization  to  enforce  its  policies.  The 
relationships  between  line,  service,  staff,  and  policy 
organizations  shall  be  identified. 

(3.3.3)  Management  and  Control :  The  program  plan  shall 
include  detailed  listing  of  specific  tasks,  man- loading  per 
task,  and  procedures  to  implement  and  c  ntrol  these  tasks. 

It  shall  include  a  description  of  each  task  to  be  performed 
whether  or  not  it  is  already  documented  in  contractor 
directives,  the  organizational  unit  with  the  authority  and 
responsibility  for  executing  each  task,  the  method  of  control 
to  insure  executive  of  each  task  as  planned,  and  scheduled 
start  and  completion  dates  of  each  task.  This  data  shall  be 
in  a  form  that  permits  technical  auditing  by  the  procuring 
activity.  The  information  provided  shall  include  the  method 
of  analysis  to  be  used  as  a  basis  for  achieving  the  proper 
balance  of  effort  and  resources  from  a  reliability  standpoint. 
The  contractor  shall  identify  specific  technical  problems 

to  be  solved,  review  problems  considering  program  require- 
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merits,  and  develop  a  detailed  program  to  solve  the  problems. 
Records  shall  be  maintained  on  the  status  of  actions  to 
resolve  problems.  All  designers  and  associated  personnel 
shall  be  made  aware  of  the  reliability  requirements  per¬ 
taining  to  their  area  of  responsibility  and  shall  be  in¬ 
cluded  in  the  information  loop  to  correct  known  deficiencies. 
The  designation  of  milestones,  definition  of  inter-relation¬ 
ships,  and  estimation  of  times  required  for  reliability  pro¬ 
gram  activities  and  tasks  shall  be  employed  as  part  of  over¬ 
all  program  control  which  applies  the  program  techniques. 

If  PERT  (Program  Evaluation  and  Review  Techniaues)  is  part 
of  the  program  it  shall  be  utilized. 

MIL-R-227 32B  (3.2)  Reliability  Assurance  Program;  The  con¬ 
tractor  shall  establish  and  conduct  a  reliability  assurance 
program  including,  as  a  minimum,  the  elements  required  by  this 
#pecif ication.  The  contractor's  reliability  assurance  program 
shall  be  consistent  with  the  requirements  of  MIL-STD-441  and 
the  requirements  of  this  specification.  Where  the  requirements 
of  MIL-STD-441  and  this  specification  conflict,  the  requirement* 
of  this  specification  shall  govern.  The  fundamental  features  of 
the  reliability  assuiance  program  shall  be  consistent  with  the 
extent  to  which  the  particular  procurement  embraces  the  pro¬ 
curement  phases  of  feasibility  study,  design  and  development, 
prototype  (preproduction)  fabrication,  and  production.  When 
these  phases,  either  individually  or  collectively,  are  included 
in  the  procurement  the  reliability  program  elements  listed  there¬ 
under  are  required. 

3.2.4  Evaluation  Tasks:  The  treatment  afforded  by  MIL-R-22732 
(paragraph  3.2.2  and  4.3)  in  the  use  of  reliability  analysis  to 
improve  the  product  while  in  the  design  stage  is  excellent. 
Reports  are  not  specified  in  detail.  The  visibility  would  be 
improved  by  some  elaboration  of  reporting  requirements. 

MIL  -STD-78  5  (3.2.2)  Reliability  Requirement  Studies;  The  relia¬ 
bility  program  shall  procide  for  preliminary  and  continuing 
studies  of  reliability  estimates  and  achievements.  The  relia¬ 
bility  program  for  all  program  phases  shall  provide  for  pro¬ 
gressive  refinement  of  the  reliability  analysis  and  validation 
of  specified  requirements  for  all  planned  missions  or  operational 
modes  of  the  system.  These  studies  shall  include  definition  of 
functional  performance  limits,  duration  of  operation  in  time  or 
cycles,  etc.,  and  the  environmental  conditions  of  operational 
use.  Apportionment  of  reliability  requirement  from  the  system 
to  system  elements  shall  consider  complexity  and  importance 
(effect  of  failure)  of  the  system  elements  including  alternative 
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modes  of  operation.  Progressive  reliability  goals  shall  be 
established  for  each  maior  phase  of  a  program  which  are  phased 
with  program  review  points  (3.4}. 


(3.5.4)  Critical  Items:  The  contractor  shall  establish  an 
effective  method  for  identification,  control  and  special  handling 
of  critical  parrs,  components,  subsystems  or  other  end  items  from 
design  through  final  acceptance.  Such  methods  shall  be  described 
in  the  contractor's  formal  policies  and  procedures  to  assure 
awareness  by  all  affected  personnel  (e.g.,  design,  purchasing, 
manufacturing,  inspection,  test,  handling,  etc.)  of  the  essential 
and  critica-  nature  of  such  items.  The  methodology  used  in 
generating  the  critical  item  li„t  shall  be  furnished  to  the  pro¬ 
curing  activity.  The  method  used  and  the  list  subsequently 
generated  shall  be  subject  to  review  and  evaluation  of  the  pro¬ 
curing  activity. 

(3.5.5)  Mathematical  Models;  The  contractor  shall  provide 
mathematical  models  based  on  systems  analysis  to  apportion  relia¬ 
bility  over  major  systems  elements;  and  to  predict  reliability 

at  various  stages  of  design.  The  mathematical  models,  apportion¬ 
ment,  and  initial  prediction  shall  be  included  in  the  program 
plan 

(3.5.9)  Human  Engineering;  The  reliability  program  shall  apply 
the  principles  of  human  engineering  in  all  operations  during 
design  development,  manufacture,  test,  maintenance,  and  oper¬ 
ation  of  the  system  or  subsystem.  The  design  sha^l  incorporate 
human  engineering  features  that  minimize  the  possibility  of  de¬ 
grading  reliability  through  human  error.  Contractor's  human 
engineering  personnel  shall  participate  in  design  activity  and 
proposed  tests  to  assure  that  the  principles  in  MIL-STD-803  have 
been  incorporated  in  design  and  are  reflected  in  test  plans. 

(3.5.10)  Statistical  Methods:  The  contractor's  reliability 
program  shall  incorporate  optimum  utilization  of  statistical 
planning  -nd  analysis.  This  shall  include  application  of  such 
methods  as  design  of  experiment,  analysis  of  variance,  and  other 
methods  applicable  to  design,  development,  and  production  phases. 

(3.5.11)  Maintainability:  The  effects  of  the  reliability  pro¬ 
gram  on  the  maintainability  of  the  design  shall  be  considered 
during  the  initial  nd  subsequent  design  phases  to  assure 
minimum  degradation  to  system  availability. 

( 3 . 5 . 1 7 )  Effects  of  Storage,  Shelf-Life,  Packaging,  Transpor  t .  - 
ion,  Handling,  and  Maintenance;  The  contractor  shall  determ, 
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by  test  and  analysis,  or  shall  estimate,  the  effects  of  storage, 
shelf-life,  packaging,  transportation,  handling  and  maintenance 
on  the  reliability  of  the  product.  He  shall  design  the  product 
to  withstand  these  effects.  Any  special  requirements  or  limita¬ 
tions  on  shelf-life,  storage,  packaging,  transportation,  handling, 
and  maintenance  shall  be  made  known  to  the  procuring  activity. 

MIL-R-227 32B  (3.2.1)  Feasibility  Studies:  Reliability  she1!  bo 
considered  in  determining  whether  or  not  practical  apulicati  n 
of  a  concept  or  tentative  design  is  possible  for  this  purpose, 
an  estimate  of  reliability  shall  be  determined  using  the  appli 
cable  method  set  forth  in  NAVSHIPS  93820  consistent  with  the 
extent  to  which  the  design  configuration  is  known.  The  effect 
of  the  estimated  reliability  on  other  mission  parameters  of 
maintainability,  availability,  .and  effectiveness  shall  b 
analyzed  and  identified  together  with  a  description  of  the 
relationships  between  reliability  and  these  parameters  i 
achieving  the  intended  mission  objective. 

(3.2.2)  Design  and  Develoi  ae-  I*"  shall  be  recognized  by  the 
contractor  that  the  inherent  el  lability  of  any  product  is 
determined  by  the  basic  design  '  nich  is  the  limiting  factor  i  ' 
achieving  high  reliability  during  military  use.  Accordingly, 
the  major  emphasis  by  the  contractor  in  attaining  the  degree  of 
reliability  required  by  the  individual  equipment  specification 
must  be  applied  during  product  design  and  development.  In 
product  design  and  development  the  following  reliability  program 
elements  are  required. 

(3.2.2. 1)  Product  Identification:  The  contractor  >nall 
identify  the  complete  product  involved  in  the  procurement 
to  which  the  numerical  reliability  requirement  in  the  in¬ 
dividual  equipment  specification  applies.  The  mission 
objective  shall  be  delineated  together  with  the  specific 
criteria  for  determining  product  success  or  failure.  The 
numerical  reliability  requirement  shall  be  interpreted  by 
the  contractor  in  terms  of  the  mission  objective,  the 
product  configuration,  and  the  criteria  for  success  and 
fai lure. 


(3. 2. 2. 2)  Reliability  Design  Guides:  The  contractor  and 
his  personnel  shall  familiarize  themselves  with  the  Bureau 
of  Ships  Reliability  Design  handbook  NAVSHIPS  and  make 
maximum  use  of  the  design  guides  therein  in  the  design  and 
design  modifications  required  by  the  individual  equipment 
specification  as  well  as  this  specification. 
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(3. 2. 2. 3)  Mathematical  Model:  The  contractor  shall  establish 
a  mathematical  model  relating  the  reliability  of  the  complete 
product  to  the  design  configuration,  modes  of  operation,  duty 
cycles,  and  reliability  indexes  used  for  evaluation.  The 
mathematical  model  shall  provide  the  basis  for  reliability 
prediction,  analytical  reliability  assessment,  and  allocation 
of  reliability  goals  to  lower  levels  within  the  product. 

(3. 2. 2. 4)  Allocation  of  Reliability  Requirements:  The  con¬ 
tractor  shall  apportion  the  reliability  requirement  from  the 
individual  equipment  specification  to  lower  levels  within  the 
product  by  allocating  numerical  reliability  goals  to  each  sub¬ 
system,  equipment,  assembly,  sub-assembly,  down  to  each  non- 
repairable  part.  When  recombined  in  accordance  with  the 
mathematical  model  (see  3. 2. 2. 3)  the  allocated  goals  shall 
yield  a  product  reliability  which  equals  or  exceeds  the  re¬ 
quirement  in  the  individual  equipment  specification.  The 
detail  goals  shall  provide  the  basis  for  establishing  relia¬ 
bility  criteria  for  suppliers  products  and  for  evaluating 
progress  when  compared  with  the  results  of  subsequent  relia¬ 
bility  predictions  and  tests.  Such  comparisons  shall  serve 

as  a  means  for  detecting  potential  trouble  areas  and  for 
adjusting  reliability  effort  to  areas  where  needed  to  meet 
required  reliability  levels. 

(3. 2. 2. 5)  Initial  Reliability  Prediction;  The  contractor 
■hall  perform  an  initial  reliability  prediction  for  the 
complete  product  utilizing  Method  C  set  forth  in  NAVSHIPS 
93820.  Parts  or  subunits  not  covered  by  existing  data  in 
NAVSHIPS  93820  shall  be  identified  and  means  for  obtaining 
reliability  figures  of  merit  for  these  items  shall  be  stated 
by  the  contractor.  Use  of  reliability  data  from  other  sources 
su  'h  as  parts  suppliers  or  other  reliability  documentation  is 
permissible  subject  to  approval  by  the  procuring  agency,  how¬ 
ever  ,  such  data  shall  not  take  precedence  over  data  for 
identical  items  contained  in  NAVSHIPS  93820  unless  fully 
justified  by  the  contractor  and  approved  by  the  procuring 
agency.  Correlation  shall  be  made  between  allocated  relia¬ 
bility  qoals  (see  3. 2. 2. 4)  and  reliability  predictions. 

(3. 2. 2. 7)  Final  Reliability  Prediction;  The  contractor  shall 
perform  a  final  reliability  prediction  incorporating  all  design 
changes  made  during  the  development  process  and  representing 
the  final  uesign  configuration  to  be  used  in  the  product 
Method  D  of  NAVSHIPS  33820  shall  be  used  together  with  data 
from  other  sources  as  required  and  substantiated,  taking  into 
account  the  failure  characteristics  of  parts  for  which 
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severity  (stress)  fun^  ions  versus  failure  rate  are  known  and 
documented.  Correlation  shall  be  made  between  allocated  relia¬ 
bility  goals  (see  3. 2. 2. 4)  and  the  final  reliability  predict¬ 
ion. 

(3.2.8)  Qualitative  Requirements:  Equipments  that  do  not  have 
a  specified  minimum  MTBF  shall  be  designed  and  produced  to  attain 
the  maximum  practical  MTBF.  Such  equipment  shall  be  free  of  the 
known  causes  of  poor  reliability  such  as  unnecessary  complexity, 
misapplication  of  parts,  marginal  design,  and  poor  workmanship. 

The  reliability  assurance  procedure  prescribed  in  4.3  shall  be 
applied  to  veri fy  that  the  equipment  fulfills  this  requirement. 

(4.3)  Reliability  Assurance  by  Analysis  and  Prediction:  The 
following  procedure  shall  be  applied  to  all  electronic  equip¬ 
ments  regardless  of  whether  quantitative  reliability  require¬ 
ment  are  involved  and  regardless  of  whether  reliability  testing 
is  required. 

(a)  The  MTBF  of  the  equipment  shall  be  predicted  using 
Method  D  of  publication  NAVSHIPS  93820. 

(b)  The  design  shall  be  reviewed  and  analyzed  in  detail  by 

a  group  provided  by  the  contractor  independent  of  the  design¬ 
ers  to  determine  that  it  is  inherently  as  reliable  as  is 
practical.  It  shall  be  the  particular  function  of  this  group 
to  constructively  criticize  such  common  weaknesses  as  un¬ 
necessary  complexity,  misapplication  of  part<,,  and  those 
commonly  called  "marginal  design".  This  group  shall  report 
the  results  of  the  design  review  and  analysis  together  with 
recommendations  to  the  procurement  agency  and  the  designers. 

(c)  Any  failure  of  a  prototype  or  preproduct ioi.  equipment 
that  occurs  during  the  development,  construction,  or  testing 
of  the  equipment  shall  be  analyzed  and  reported  to  the  pro¬ 
curement  agency.  The  analysis  shall  be  conducted  in  such  a 
manner  as  to  determine  the  causi  of  the  failure  so  that  its 
recurrence  can  be  prevented.  Reports  of  the  failures  >nd  their 
analyses  shall  bt.  forwarded  to  the  design  review  group  for 
endorsement . 

The  equipment  shall  be  considered  acceptable  whenever  the  relia¬ 
bility  prediction,  the  design  review,  and  the  failure  analyses 
are  completed  and  the  procurement  agency  is  satisfied  that  any 
faults  revealed  by  these  studies  have  been  corrected. 
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3.2.5  Design  Review : 

M IL-STD-78  5  (3.5.b.l)  Periodic  design  reviews  for  reliability 
and  evaluation  of  designs  shall  be  conducted  as  an  integral  part 
of  the  contractor's  engineering  design  review  and  evaluation 
procedures.  These  reviews  shall  evaluate  the  achievement  of 
reliability  relative  to  the  reliability  goals  established  for 
each  major  phase  and  review  point  of  the  contract;  with  con¬ 
tractor  evaluation  before  designs  are  finalized.  The  relia¬ 
bility  design  review  analyses  shall  include,  to  the  extent 
applicable : 

(1)  Reliability  estimates  based  upon  prediction  (such  as 
MIL-STD-756  and  MIL-HDBK-217  as  basic  data)  and  accumulated 
test  data.  Estimates  shall  be  made  for  each  mode  of  oper¬ 
ation. 

(2)  Review  of  potential  design  or  production  problem  areas. 

(3)  Analysis  of  effects  of  failure. 

(4)  Identification  of  the  principle,  critical  items  in¬ 
hibiting  reliability  achievement. 

(5)  The  effects  of  engineering  decisions  and  trade-offs 
upon  reliability  achievements,  potential  and  growth. 

(3.3  .6.2)  The  program  plan  shall  specify  appropriate  personnel 
from  the  contractor's  reliability  organizations  who  shall  parti¬ 
cipate  in  the  design  reviews  and  denote  approval  by  signature. 
These  review-s  shell  be  continuing  in  nature  to  provide  for  the 
earlie  t  possible  detection  and  correction  of  any  potential 
deficiencies.  A  system  shall  be  established  and  ma  ntained  by 
the  contractor  to  assure  reliability  participation  in  control 
of  designs,  specifications,  drawings,  and  all  changes  thereto. 

(3. 5. 6. 3)  The  design  review  shall  compare  the  design  with 
previously  defined  qualitative  and  quantitative  requirements. 

The  results  of  the  review  shall  be  documented. 

(3. 5.6.4)  The  procuring  activity  shall  be  notified  at  least 
10  days  prioi  to  each  scheduled  formal  design  review  (as  dis¬ 
tinguished  from  continuing),  to  permit  procuring  activity 
participation.  The  minutes  of  such  reviews  shall  be  made 
available  to  the  procuring  activity  upon  reques*. 
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MIL-R-227 32B  (3*2. 2. 6)  Reliability  Design  Review*  The  con¬ 

tractor  shall  perform  a  reliability  design  r evi ew“f unction  con¬ 
ducted  by  experienced  reliability  personnel  not  directly  sub¬ 
ordinate  to  the  design  engineering  function.  The  design  review 
shall  include  a  detailed  examination  of  the  design  documents, 
drawings,  ard  specifications  and  a  complete  evaluation  of  the 
effects  of  r-u"t  selection  and  application  on  the  reliability  of 
the  product.  Evaluations  shall  include  analysis  of  environmental 
•tresses  (temperature,  humidity,  vibration)  as  well  as  physical 
stresses  (electrical,  mechanical)  sustained  by  parts  during  in¬ 
tended  military  use  of  the  product.  Identification  shall  be  made 
of  critical  or  marginal  features  of  the  product  desi  gr.'  which 
adversely  affect  reliability  as  determined  by  the  design  review. 
Provision  s  all  be  made  by  the  contractor  for  approval  by  the 
reliability  design  review  function  prior  to  final  release  of  the 
product  design. 

(3.2.3)  Incompatibility  of  existing  design  and  required  relia¬ 
bility;  In  the  event  that  incompatibility  of  existing  design 
and  required  reliability  is  established,  the  contractor  shall 
prepare  and  submit  to  the  procuring  activity  for  approval,  a 
proposed  program  for  accomplishing  such  design  changes  as  are 
required  to  insure  compatibility.  Where  possible,  such  design 
changes  shall  include  at  least,  but  shall  not  be  limited  to 
consideration  of  reduction  of  thermal  and  electrical  stresses 
on  the  equipment  part  implement.  When  it  is  established  that 
the  required  reliability  level  is  not  obtainable  within  the 
existing  state  of  the  art  for  parts,  the  contractor's  proposal 
shall  include  a  diagnosis  of  optimum  types  of  and  locations  for 
redundant  circuitry  and  the  scope  of  the  work  necessary  to 
provide  the  required  reliability  by  these  means. 

3.2.6  Parts  Reliability x  MIL-STD-785  provides  a  very 
effective  coverage  of  parts  control.  MIL-R-227  3-2B  does  not 
cov^r  this  area.  Where  MIL-E-16400  is  utilized  in  the  procure¬ 
ment,  the  parts  selection  requirement  requires  selection  of  stan¬ 
dard  parts  from  MIL-STD-242.  These  parts  are  preferred  on  a 
basis  of  standardization  rather  than  on  a  basis  of  high  relia¬ 
bility  ar.d  hence  do  not  guarantee  performance  of  the  part. 

MIL-STD-785  (3.5.3)  Parts  Reliability:  Parts  shall  not  be  used 
without  knowledge  of  their  capabilities  and  reliability  potential 
determined  from  current  or  previous  testing.  Information  shall 
be  sought  or  generated:  on  stress  levels  and  limits  of  application 
as  well  as  on  failure  rate.  Available  data  and  central  in¬ 
formation  facilities  shall  be  utilized  to  avoid  hoed less  dupli¬ 
cation  of  testing.  In  using  existing  data,  the  risk  and  limit- 
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o'  determination  of  fai kare  rate  for  each  part  type  shall  be 
made;  the  part  vendor's  accumulated  test  history  under  part 
specif i  it. ions  requiring  failure  rate  verification  shall  be 
sought.  Reported  measure  of  achieved  reliability  should  not  be 
based  upon  short  duration  tests  which  predominately  measure 
performance.  If  time  does  not  permit  adequate  testing  at 
advanced  ages,  the  contractor  shall  show  the  age  range  actually 
tested  and  shall  justify  use  of  such  data. 

{3. 5. 3.1}  Where  estimates,  data,  and  experience  indicate  a 
need  for  a  parts  reliability  improvement  program  to  achieve 
desired  system  reliability,  the  contractor  shall  propose  a 
program  to  increase  the  standardization  and  reliability  of 
parts  to  the  required  level.  A  preferred  parts  list  shall  be 
maintained  and  utilized  as  a  source  of  high  reliability  parts. 

(3. 5. 3. 2)  Emergency  Reporting  of  Defective  Parts:  When  a 
MIL  specification  or  a  MIL  part  is  deemed  suspect  by  the 
development  contractor,  the  contractor  shall: 

(a)  Indicate  reason  with  supporting  evidence  of  this 
conclusion. 

(b)  Perform  fai.  ^d  part  diagnosis  and  analysis  of  those 
parts  deemed  suspect  development,  acceptance  tests,  and 
other  related  activities. 

(c)  Whenever  possible,  reach  a  conclusion  relative  to 
the  cause  of  failure. 

(d)  Report  by  most  expeditious  means  to  the  procuring 
activity  with  concise  supporting  data  when,  and  only  when, 
it  has  been  concluded  that  a  part  is  unsatisfactory  for 
any  of  the  following  reasons x 

(1)  A  part  which  was  accepted  as  meeting  a  MIL 
specification  but  which  failed  to  perform  to  expecta¬ 
tions,  such  failure  concluded  to  be  attributable  to: 

(a)  Manufacturing  procedures,  choice  of  materials,  or 
design  of  part,  or  (b)  Test  and  inspection  disciplines. 

(2)  A  military  part  specification  which  is  inadequate 
in  that  it  (a)  does  not  take  advantage  of  the  state-of- 
the-art,  (b)  requires  amendment  to  encourage  advanoe- 

i 

i 


ment  of  the  state-of-the-art;  or  (c)  requires  revision 
for  clarity. 

3.2.7  Supplier  Control; 

MIL-STD-78 5  (3.5.7)  Supplier  and  Subcontractor  Reliability 
Programs :  The  contractor  shall  be  responsible  for  assuring  that 
suppliers's  and  subcontractor's  achieved  reliability  levels  are 
consistent  with  overall  system  requirements.  The  contractor 
shall  impose,  directly  or  indirectly,  quantitative  reliability 
requirements  and  acceptance  criteria  on  all  echelons  of  suppliers 
and  subcontractors;  and  shall  incorporate  applicable  portions  cZ 
this  standard  in  subcontracts  and  purchase  orders.  The  relia¬ 
bility  program  of  the  contractor  shall  contain  provisions  for 
surveillance  of  supplier  and  subcontractor  reliability  activities 
including  failure  reporting.  The  surveillance  shall  consist  of 
but  not  be  limited  to  such  items  as  maintaining  a  supplier 
selection  program  based  upon  review  of  the  supplier’s  reliability 
program,  quality  control  system,  examination  of  his  facilities, 
and  past  performance,  to  assure  that  suppliers  are  capable  of 
attaining  and  maintaining  the  required  level  of  reliability.  The 
contractor  shall  take  .1  actions  necessary  to  assure  that  no 
changes  made  by  any  supplier  will  reduce  reliability  of  the 
system.  Records  of  each  supplier's  performance  shall  be  main¬ 
tained  and  reviewed  with  him  periodically. 

3.2.8  Failure  Data  and  Diagnosis:  MIL-STD-785  provides  for  the 
collection  of  success/ failure  data  and  the  analysis  of  failures 
occurring  to  the  product  or  to  the  components  before  assembly. 
MIL-R-22732  requires  recording  of  failures  occurring  during  'die 
reliability  tests,  with  analysis  and  report  of  their  causes. 

MIL-STD-78 5  (3.5.15)  Failure  Data  Collection,  Analysis,  and 
Corrective  Action?  (a).  The  contractor  shall  have  and  shall 
require  major  subcot  tractors  to  have  a  closed  loop  system  for 
collecting,  analyzing,  and  recording  all  failures  that  occur  during 
phases  of  teqts  required  for  system  elements  including  those  that 
are  performed  in-plant  and  at  installation  sites.  The  contractor 
shall  describe  his  failure  reporting  procedures,  including  flow 
charts,  for  the  analysis,  feedback  and  corrective  action  as  part 
of  the  program  plan  (see  3.3.3).  The  contractor  shall  explain 
the  method  by  which  failure  reports  are  initiated.  Analysis  and 
recording  of  failures  shall  differentiate  between,  but  not  be 
restricted  to,  those  due  to  equipment  failure  and  those  due. to 
human  error  in  designing,  processing,  handling,  transporting, 
storing,  maintaining,  and  operating  the  .equipment .  Elapsed  time 
indicators  on  event  counters  shall  be  utilized  or  a  log  shall  be 
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maintained  to  report  accumulated  operation  time  or  operation 
cycles  on  system  components  that  arc  time  or  operation  cycle 
sensitive,  The  failure  reporting  system  shall  be  designed  to 
fce  compatible  with  the  maintenance  data  collection^system  of  the 
procuring  or  using  activity  so  that,  as  the  system. nears  the 
operational  inventory  phase,  transistion  to  in-service  failure 
reporting  can  be  accomplished  with  the  minimum  disturbance  and 
maximum  continuity  of  effort.  The  failure  reporting  system  shall 
include  provisions  to  assure  that  effective  corrective  actions 
are  taken  on  a  timely  basis  to  reduce  or  prevent  repetition  of 
the  failures.  The  contractor  shall  establish  scheduled  audits 
to  review  all  open  reports,  analyses,  dates  for  corrective 
action  and  report  all  delinquencies  to  management. 

(b)  The  contractor  shall  commence  failure  reporting  with  initial 
development  testing  or  operation  including  operating  equipment  at 
receiving  inspection,  at  a  vendor's  plant  in  final  assembly  check¬ 
out,  or  during  acceptance  testing.  An  unscheduled  adjustment, 
other  than  a  calibration  made  during  other  maintenance  because 

of  convenience,  shall  be  defined  as  a  failure  for  reporting  pur¬ 
poses.  Failures  of  components  prior  to  incorporation  into  an 
assembly  shall  be  recorded  separately  and  reported. 

(c)  The  contractor  shall  submit  failure  report  summaries  as 
specified  by  the  procuring  activity. 

\ 

3.2.9  Supporting  Activities  t 

MIL-STD-785  (3.5.2)  Furnished  Equipment;  Where  other  equipments, 
such  as  Government- furnished  or  associate  contractor  supplied  y  > 
equipment  are  to  be  integrated  to  provide  a  complete  operational : 
system,  the  contractor  shall  use  known  or  estimated  reliability 
values  for  these  equipments.  When  such  empirical  data  are  not 
available  through  the  channels  to  which  the  contractor  has  access, 
the  contractor  shall  request  such  data  from  the  procuring 
activity.  The  contractor  shall  report  potential  reliability 
problems  introduced  by  deficient  Government- furnished  equipment 
or  other  associated  equipment  over  which  he  has  no  control  and 
shall  indicate  and  justify  the  system  changes  necessary  to 
accommodate  or  the  improvement  necessary  to  make  this  equipment 
compatible  with  the  system  requirements . 

,  •  "  ’■>  _ 7. 1 

(3.5.8)  Reliability  Indoctrination  and  Training:  The  relia¬ 
bility  program  shall  contain  provisions  to  supplement  the  basic 
training  and  indoctrination  of  company  and  plant  personnel  with 
reliability  training  to  assure  that  their  skills  and  knowledge 
keep  pace  with  advancing  technology  and  the  requirements  or 


peculiarities  of  the  system  or  equipment. 

(3.5.14)  Manufacturing  Controls  and  Monitoring :  The  c ontr actor 
shall  have  a  planned,  controlled  and  schedule  system  of  pro¬ 
duction  control  and  monitoring  to  assure  that  reliability  achieved 
in  design  is  maintained  during  production. 

MIL-R-22732B  (3.2.9)  Support  Activity:  The  contractor's  facil¬ 
ities  and  organization  shall  be  such  as  to  insure  that  all 
support  which  affects  equipment  reliability  will  be  accomplished 
in  a  manner  compatible  with  the  requirements  of  this  specifi¬ 
cation.  Such  support  shall  include,  but  is  not  limited  to,  the 
following:  (a)  Quality  control  systems  requirements  in  accord¬ 

ance  with  MIL- E-16400,  and  (b)  Reliability  indoctrination  of 
personnel. 

3.2.10  Monitoring  and  Review:  MIL-STD-7S5  provides  for  the 
program  to  be  reviewed  at.  planned  points.  MIL-R-227  32  requires 
progress  reports  monthly  and  at  the  conclusion  of  specific 
elements,  (apportionment,  prediction,  design  review  and  testing) . 
The  coverage  in  the  progress  report  is  not  clearly  specified, 
but  the  inference  would  seem  to  include  the  progress  and  results 
on  the  particular  elements. 

MIL-STD-785  (3.4)  Program  Review:  The  reliability  program  shall 
be  organized  and  scheduled  to  permit  the  contractor  and  the 
procuring  activity  to  review  its  status,  including  results 
achieved,  at  pre-planned  steps  or  checkpoints.  This  formal 
review  and  assessment  of  reliability  normally  will  be  conducted 
at  major  program  points  and  these  points  will  be  established  by 
the  procuring  activity  during  negotiations.  As  the  program 
develops,  reliability  progress  shall  be  assessed  by  use  of  such 
information  as  predictions  of  reliability  and  results  of  relia¬ 
bility  design  review.*,  and  tests  including  effects  of  human  per¬ 
formance  . 

MlL-R-22732B(3 .2 .8.1)  Reporting :  The  results  of  the  reliability 
assurance  plan  shall  be  summarized  in  a  report  to  the  procure¬ 
ment  agency.  This  report  shall  include  the  results  of  the  relia¬ 
bility  prediction,  design  review,  and  failure  analyses  specified 
in  4.3  together  with  a  discussion  of  the  information  contained 
therein . 

(3.2.4)  As  a  minimum,  reports  shall  be  submitted  45  days  after 
award  of  contract  and  monthly  thereafter  to  end  of  contract,  or 
at  the  conclusion  of  each  program  element  specified  in  3. 2. 2.1; 
3.2. 2.3,  3. 2. 2. 4,  3. 2. 2. 5,  3. 2. 2. 6,  3. 2. 2. 7,*  3. 2. 5. 2,  3. 2. 6. 2, 
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and  3*2. 8.1  whichever  occurs  more  frequently. 

3.3  MAINTAINABILITY  SPECIFICATION  ABSTRACTS 

There  .is  considerable  overlap  between  maintainability  and  relia¬ 
bility  specifications,  the  prime  example  being  design  review. 

Most  such  specifications  read  as  though  separate  design  review 
meetings  are  required  for  maintainability,  reliability,  safety, 
and  the  host  of  other  important  design  considerations.  To  the 
contractor  this  is  not  economical  or  relaistic,  since  trade-offs 
between  these  disciplines  are  involved.  For  this  reason  Chapters 
22  and  23  recommend  language  that  combines  the  discipline  Where 
it  is  appropriate. 

For  electronic  systems,  MIL-M-23313  provides  a  useful  basic 
specification  for  obtaining  a  required  MTTR  in  the  product. 

This  specification  is  not  referenced  in  the  general  electronic 
system  specification  MIL-E-16400,  nor  in  MIL-R-22732 .  To  apply 
MIL-M-23313  to  a  particular  procurement  requires  (a)  reference 
to  MIL-M-23313  in  the  procurement  document,  and  (b)  assignment 
of  quantitative  requirements. 

There  is  no  BuShip  specification  or  MIL-5TD  released  for  main¬ 
tainability  requirements  on  systems  other  than  electronic.  To 
approach  maintainability  in  such  systems,  the  specific  clauses 
must  be  developed  in  the  procurement  documents.  In  building  up 
such  a  specification,  (a)  the  numerical  requirement  must  be 
speci fifed,  (b)  the  relevant  factors  of  maintenance  philosophy, 
replacement  level  of  parts  or  components,  and  software  support 
must  be  provided,  (c)  maintainability  studies,  design  review 
effort,  predictions  and  apportionment  and  special  logistics 
studies,  should  be  required,  (d)  details  and  frequency  of  report¬ 
ing  should  be  outlined,  and  (e)  the  acceptance  test  plan  should 
be  defined. 

There  is  considerable  variation  between  the  approaches  of  the 
various  services  in  attempting  to  obtain  a  specified  maintain¬ 
ability  of  procured  equipment.  In  addition  to  the  Bureau's 
specification  for  maintainability  requirements  for  electronic 
equipment  and  systems,  selected  abstracts  of  Air  Force  and 
Bureau  of  Naval  Weapons  are  furnished  for  clarification  of 
understanding. 

3.3.1  MIL-M-23313 A  (SHIPS)  -  Maintainability  Requirements  for 

Shipboard  and  Shore  Electronic  Equipments  and  Systems. 

(1.1)  This  specification  covers  maintainability  requirements 
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for  B'ureau  of  Ships  shipboard  and  shore  electronic  equipment  and 
systems . 

(1.2)  In  addition,  this  specification  prescribes  procedures  to 
be  followed  for  evaluation  of  equipment  maintainability  during 
equipment  or  system  development  unc  production  programs.  In 
particular  it  provides  procedures  for: 

(a)  Maintainability  evaluation  of  final  design. 

(b)  Preproduction  maintainability  test. 

(c)  Maintainability  design  review  during  production. 

(d)  •  Maintainability  evaluation  during  preliminary  design. 

(3.1.1)  Maintainability  assurance  program.  -  The  contractor's 
(supplier's)  maintainability  assurance  program  shall  be  consist 
ent  with  the  requirements  of  this  specification.  The  procuring 
activity  will,  at  its  option,  review  and  evaluate  the  contractor's 
maintainability  assurance  program  to  determine  whether  or  not  it 
is  adequate  and  consistent  with  the  provisions  of  this  spec¬ 
ification.  The  maintainability  assurance  program  shall  include, 
but  shall  not  be  limited  to,  the  following  requirements  or 
applicable  portions  thereof. 

(3. 1.1.1)  Support  activity.  -  The  contractor's  facilities  or 
those  of  a  subcontractor  shall  be  such  as  to  insure  that  all 
support  which  affects  equipment  maintainability  will  be 
accomplished  in  a'  manner  compatible  with  the  requirements  of 
this  specification.  Such  support  shall  include,  but  shall 
not  be  limited  to,  the  followings 

(a)  Maintainability  indoctrination  of  personnel. 

(b)  Maintainability  design  review  throughout  the  develop¬ 
ment  and  production  program  tc  assure  that  maintainability 
is  being  considered  as  a  design  goal. 

(3. 1.1. 2)  Maintainability  design  guides.  -  The  contractor 
and  his  personnel  shall  familiarize  themselves  with  Publi¬ 
cation  NAVSHIPS  94324,  and  make  maximum  use  of  the  design 
guides  therein. 

(3. 1.1. 3)  Maintainability  during  design  and  production.  - 
The  maintainability  evaluation  procedure  and  maintainability 
test  described  in  the  appendix  shall  be  used  for  evaluating 
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the  maintainability  of  equipments  and  systems  in  the  final 
design  stage,  and  in  the  pre product ion  and  production  stages 
(see  Sections  30  and  40) . 

(3.1. 1.4)  Maintainability  prediction  during  preliminary 
design  stage.  -  In  the  preliminary  design  stage  (see  6.3.3) 
the  contractor  may  use  any  evaluation  method  and  any  schedule 
of  evaluation  suitable  to  him  to  assure  himself  of  compliance 
with  this  specification  in  the  final  design  stage  and  in  the 
preproduction  and  production  stages.  Four  evaluation  methods 
for  various  stages  of  development  and  design  are  presented  in 
the  appendix  (see  Section  50) . 

(3.2)  Maintainability  requirements .  -  The  procuring  activity 
will  specify  an  equipment  repair  time  (ERTO  in  the  detailed 
equipment  or  systems  specification.  (See  6.4)  The  design  of 
the  equipment  or  system  shall  be  such  that  the  geometric  mean 

of  all  active  repair  time  intervals  required  to  repair  independ¬ 
ent  failures  shall  not  exceed  the  specified  ERT.  Compliance 
with  this  requirement  will  be  verified  in  the  final  design  stage, 
and  5.n  the  preproduction  and  production  stages  when  the  follow¬ 
ing  criteria  are  met. 

(6.4)  Maintainability  specification  method.  -  The  value  of 
Equipment  Repair  Time  (ERT)  to  be  specified  in  the  detailed 
equipment  specification  (See  3.2)  should  be  determined  using 
the  following  expression: 

EP.T  (specified)  =0.37  ERTmax> 

where  (a)  ERTmax  “  the  maximum  value  of  ERT  that  should  be 
accepted  no  more  than  10  percent  of  the  time,  and  (b)  0.37  =  a 
value  resulting  from  application  of  "Student's  t"  operating 
characteristics  and  that  assures  a  95  percent  probability  that 
an  equipment  having  an  acceptable  ERT  will  not  be  rejected  as 
a  result  of  the  first  maintainability  test  when  the  same  size 
is  20,  and  assuming  a  population  standard  deviation  (a)  of  0.55. 

(3.2.1)  Maintainability  requirements  in  final  design  stage. 

The  contractor  will  be  considered  to  have  met  the  specified 
maintainability  requirements  in  the  final  design  stage  and  prior 
to  fabrication  of  the  preproduction  model  (see  6.3.5) ,  when  the 
calculated  geometric  mean-time-to-repair ,  determined  by  the 
maintainability  evaluation  of  the  final  design  is  not  more  than 
the  specified  ERT  (see  4.3) . 

(4.3)  Maintainability  evaluation  of  final  design.  -  Maintain- 
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ability  evaluation  of  the  final  design  shall  be  performed  before 
starting  fabrication  of  the  prepr eduction  model  (see  6.3.4).  The 
final  determination  of  the  schedule  for  performing  the  maintain- 
ability  evaluation  of  the  final  design  will  be  made  by  the  pro¬ 
curing  activity. 


(3.2.2)  Maintainability  requirements  In  the  preproduction  stage. 

-  The  contractor  will  be  considered  to  have  met  the  maintain¬ 
ability  requirements  for  the  preproduction  model  when  the  measured 
geometric  mean-time-to-repair  (MTTRq)  and  standard  deviation  (S) , 
as  determined  in  accordance  with  (4.4)  produce  the  following 
results 


log  MTTRg  *  log  ERT  +  0.397  (S) 

where  (a)  log  ERT  =  the  logarithm  of  the .Equipment  Repair  Time 
specified  in  3.2,  (b)  log  MTTRg  =  the  value  determined  in 
accordance  with  40.5.1,  and  (c)  S  *  the  value  determined,  in 
accordance  with  40.5.2. 

(4.4)  Pr epi oduction  maintainability  .test.  -  The  preproduction 
maintainability  test  shall  be  performed  on  the  preproduction 
model  (see  6.3.5)  and  before  the  start  of  production. 

(3.2.3)  Maintainability  requirement  during  production.  -  The 
contractor  will  be  considered  to  have  met  the  maintainability 
requirements  for  production  models  if  no  design  changes  or 
modifications  are  introduced  following  acceptance  of  the  pre- 
production  model  or  if  maintainability  of  the  equipment  or 
system  has  not  been  degraded  below  the  specified  ERT  (see  3.2) 
by  the  introduction  of  design  changes  or  modifications.  (See  4.5). 

(4.5)  Maintainability  design  review  during  production.  -  When¬ 
ever  design  change (s)  are  proposed  for  any  reason  during  pro¬ 
duction  and  when  so  directed  by  the  procuring  activity,  the  con¬ 
tractor  shall  review  the  proposed  change (s)  to  assure  that  the 
overall  maintainability  of  the  equipment  will  not  be  degraded  as 
&  result  of  the  change(s).  This  review  shall  include  a  des¬ 
cription  of  the  proposed  change (s)  and  a  revision  of  the  main¬ 
tainability  evaluation  of  final  design  of  (4.3)  reflecting  the 
overall  effect  of  the  change(s).  If,  in  the  opinion  of  the 
procuring  activity,  there  is  a  possibility  that  degradation  of 
the  overall  maintainability  will  occur  as  a  result  of  the  design 
change (8),  the  procuring  activity  reserves  the  right  to  require 
a  maintainability  test  to  be  performed  on  a  production  model 
incorporating  the  design  change (s)  to  determine  the  extent  of 
such  degradation.  The  particular  model  to  be  tested  will  be 
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selected  by  the  procuring  activity,  '•■•hen  such  test  is  required 
by  the  procuring  activity,  the  t-pst  snail  be  performed  by  the 
contractor  and  in  accordance  with  section  (40)  of  the  appendix. 
Acceptance  criteria  for  this  test  shall  be  the  same  as  for  the 
preproduct  ion  maintainability  test  (see  3.2.2),  with  the  non- 
compliance  provisions  of  (4.4.2)  applicable.  When  the  test 
demonstrates  that  the  equipment  maintainability  does  not  meet 
specified  maintainability  requirements,  production  shall  be  sus¬ 
pended  pending  coordination  between  the  contractor  and  procuring 
activity  to  resolve  problem  areas  in  the  design. 

(3.3.1)  Anticipated  nonconformance.  -  In  the  event  the  con¬ 
tractor's  maintainability  evaluations  during  the  preliminary 
design  stage  indicate  that  the  specified  maintainability  will 
not  be  obtained  within  the  existing  state-of-the-art,  the  con¬ 
tractor  shall  submit  a  report  to  the  procuring  activity  explain¬ 
ing  : 

(a)  The  reasons  why  the  specified  maintainability  cannot  be 
obta i ned , 

(b)  The  -pecific  level  of  maintainability  that  can  be 
achieved;  and, 

(c^  The  design  changes  necessary  for  achieving  this  level 
of  maintainability. 

(3.3.2)  Nonconformance  o f  final  des ign ■  -  In  the  event  the  cal¬ 
culated  ueometi.  ic  nean-t  ime-t  e-repair  of  the  final  design  does 
not  meet  the  requirement  of  3.2.1,  the  contractor  shall  prepare 
and  submit  to  the  procuring  activity  for  approval,  a  proposed 
program  for  aecompl ishinw  such  design  changes  as  are  required 

to  insure  that  the  maintainability  requirement  of  the  final 
design  will  be  met.  Implementation  of  the  proposed  design 
changes  approved  by  the  procuring  activity  will  be  in  accordance 
with  the  terms  and  conditions  of  the  contract. 

(3.3.3)  Noncon  form  a  nee  of  pre-product  ion  model.  -  In  the  event 
the  equiprtu.it  or  systems  fail  to  meet  the  requirements  of  3.2.2 
after  the  second  maintainability  test  (see  4.4.2),  the  con¬ 
tractor  shall  effect  such  modi  float  ions  as  are  considered 
necessary  by  the  procuring  activity  to  assure  compliance. 
Followim  such  modifications,  the  test  of  4.4  shall  be  repeated. 

,  (3.3.4)  Nonconformance  of  production  model.  -  Tn  the  event  the 

*  equipment  or  system  fai ’ '  to  meet  the  requirements  of  3.2.3  the 

contractor  shall  suspend  production  pending  coordination  with 


n-44 


the  procuring  activity 


problem  areas  in  th< 


)s  iqn . 


(3.4)  Repoi ts .  -  The  cent 
evaluation  and  test  report 
(3.4.1)  Final  design  maint 


r  act  or  s  h  all  s  ubm  i  t  m  a  i  n  t  a  inabili  ty 
s  to  the  procuring  activity  as  follows: 
ainability  evaluation  report,  (3.4.2) 


Preproduct  ion  maintainability  test  report, 


(3.4.3 ) 


:uct  ion 


maintainability  review  reports. 


(3.6)  Maintainability  test  technician.  -  Unless  otherwise  spec¬ 
ified  in  the  contract,  or  by  the  Contracting  Officer  after  award 
of  the  contract,  the  procuring  activity  will  provide  an  Electronic 
Techni -ian  who  will  perform  the  test  repair  actions  of  the  main¬ 
tainability  test  described  in  Section  40  of  the  appendix.  When , 
in  the  interest  of  the  procuring  activity,  the  contractor  is  re¬ 
quired  to  furnish  a  technician  for  performances  of  the  test 
repair  actions,  the  contractor  will  be  so  notified  not  later 
than  30  days  prior  to  the  date  schedule  for  the  start  of  the 
test . 


(4.1)  Responsibility  for  inspection.  -  Unless  otherwise  specified 
in  the  contract  or  purchase  order,  the  supplier  is  responsible 

for  tv'c>  performance  of  all  inspection  requirements  as  speci  fled 
herein.  Except  as  otherwise  specified,  the  supplier  may  utilize 
his  own  i.,.ciiities  or  any  commercial  laboratory  acceptable  to  the 
Government.  The  Government  reserves  the  right  to  perform  any  of 
the  inspections  set  forth  in  the  specification,  where  such  in¬ 
spections  are  deemed  necessary  to  assure  supplies  ar d  services 
conform  to  prescribed  requirements. 

(4.2)  Classification  of  inspection.  -  Maintainability  inspection 
shall  be  classified  as  follows:  (a)  maintainability  evaluation 
of  final  design  (see  4.3),  (b)  preproduction  maintainability 
test  (see  4.4),  and  (c)  maintainability  design  review  during 
production  (see  4.5). 

(6.5.2)  Maintainability  Specification  Criterion.  -  The  criterion 
(ERT  -  0.37  ERTmax)  given  in  6.4  establishes  an  equipment  repair 
time  value  to  be  specified  as  the  maintainability  requirement  by 
the  writer  of  the  detailed  equipment  or  system  specification. 

It  is  based  on  establishing  a  maximum  acceptable  (upper  limit) 
value  of  ERTmax  from  knowm  operational  or  availability  require¬ 
ments  ,. and  determining  from  this,  an  AQL  of  specified  ERT.  This 
specified  value  of  ERT  established  by  the  criterion  of  6.4  is  such 
that  if  the  maintainability  test  resulted  in  a  measured  MTTRq  at 
exactly  the  acceptance  limit  (that  is,  log  MTTRq  -  log  ERT  +0.397 
(S))  the  second  time  the  tesc  of  20  repair  time  measurements  was 
I’erformed,  and  after  failing  the  first  test,  there  will  be  a  90 
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prrcent  probability  that  the  equipments  or  system  has  a  true 
MTTRg  less  than  ERTmax. 

-'6 .5.2. 3)  Therefor?,  the  specification  writer  should  first 
establish  the  ERTrax  (based  on  known  operational  requirements, 
and  so  forth)  which  cannot  be  accepted  more  than  10  percent  of 
the  time.  The  specified  value  of  ERT  to  be  included  in  the 
detailed  equipment  or  system  specification  is  determined  from 
ORl’m e x  using  expression  in  paragraph  6.4. 

3.3.2  WR-30,  Integrated  Maintenance  Management  for  Aeronautical 

Weapons.  Weapon  Systems  and  Related  Equipment,  Bureau  of  Naval 
Weapons . 

(1.1)  Thj.s  document  establishes  the  policy,  terms  and  conditions 
governing  the  implementation  and  execution  of  an  integrated  main¬ 
tainability  and  support  program  for  weapons,  weapon  systems  and 
related  equipments  to  be  procured  under  the  contract  in  which  this 
document  is  cited.  It  is  the  specific  intent  of  this  document  to 
charter  the  Integrated  Ma  .ntenance  Management  Team  to  manage  -  he 
total  Logistic  Support  Program.  Accordingly,  this  document  is 
designed  to  develop,  early  in  a  program,  a  maintenance  plan  which 
is  tailored  to  specific  commodities  and  contracts. 

(3.1.1)  Organization.  -  In  order  to  satisfy  the  overall  objective 
of  the  requirements  contained  herein,  it  is  essential  that  the 
Government  and  the  contractor  each  establish  an  organization  to 
achieve  the  integration  and  management  of  maintenance  resources. 

In  recognition  of  this  requirement,  the  contractor  shall  establish 
an  appropriate  organization  with  expressed  authority  and  res¬ 
ponsibility  for  responding  to  such  requirements. 

(3.1.2)  Management  Team  Establishment  and  Composition.  Within 
thirty  days  subsequent  to  award  of  contract  wherein  this  document 
is  cited,  the  Government  will  establish  an  Integrated  Maintenance 
Management  Team.  The  composition  of  the  Management  Team  will 
include  the  contractor  and  Government  personnel  responsible  for 
specific  elements  of  this  document. 

(3.1.3)  Planning  Conference.  -  At  a  date  mutually  acceptable  to 
both  the  Government  and  the  contractor,  buu  in  any  event  not 
later  than  60  days  subsequent  to  the  establishment  of  the  Inte¬ 
grated  Maintenance  Management  Team,  the  Government  will  convene 
the  Team  for  a  planning  conference  for  the  purpose  of  reviewing, 
modifying,  and  approving  the  contrp^or 1  s  detailed  plans  for 
satisfying  the  requirements  of  this  document. 
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(3.1.6)  Integrated  Maintenance  Management  Plan.  -  The  con¬ 
tractor's  documented  Integrated  Maintenance  Management  Plan  shall 
be  presented  for  approval  at  the  planning  conference.  The  plan 
shall  contain  the  following  as  a  minimum:  (a)  Management  Program 
Section,  (b)  Maintainability  Program  Section,  (c)  Personnel  and 
Training  Program  Section,  (d)  Publication  Program  Section,  (e) 
Augmented  Support  Program  Section,  (f)  Government  Support  Program 
Section,  and  (g)  Facility  Requirements  Program  Section. 

(3.1.7)  Maintenance  Engineering  Analysis  Record  (MEAR) .  -  The 
MEAR  shall  be  utilized  for  management  and  control  of  the  main¬ 
tainability  program  and  the  integration  ' f  maintenance  resources. 
The  contractor  shall  prepare  MEAR 1 s  in  accordance  with  Appendix 

A  during  the  development  program  and  complete  them  on  a  con¬ 
tinuing  basis  in  accordance  with  the  schedule  agreed  upon  during 
the  planning  conference.  MEAR’s  shall  be  prepared  for  the  end 
article;  functional  systems,  assemblies;  equipment  type  items 
which  are  programmed  for  independent  overhaul,  repair  or  parts 
replacement;  designated  special  support  equipment;  and  other 
items  to  such  range  and  depth  as  considered  necessary  to  insure 
adequate  maintenance  resource  support. 

(3.1.9)  Contractor  Maintenance  and  Failure  Data  Collection.  - 
The  contractor  shall  establish  and  implement  a  data  collection 
system  which  will  be  compatible  with  the  data  collection  program 
in  effect  by  the  using  activity.  The  contractor  will  commence 
data  collection  at  that  point  in  time  when  hardware  is  in  exist¬ 
ence  and  continued  throughout  the  Augmented  Support  Program. 

(3. 2. 2. 4)  Quality  of  Design.  -  During  the  design  phase,  the 
following  maintainability  objeclives  and  related  technical, 
economic  and  operational  constraints  shall  be  considered  as  a 
minimum  to  determine  the  optimum  manner  of  satisfying  the  main¬ 
tainability  requirements  for  the  end  articles. 

(a)  Design  so  that  the  mean  time  to  accomplish  scheduled 
and  unscheduled  maintenance  is  within  the  target  objectives 
which  must  be  met  to  satisfy  the  operational  plan  for  use. 

(b)  Design  to  minimize  the  complexity  of  frequently  per¬ 
formed  maintenance  tasks  (for  example:  servicing,  calibrat¬ 
ion,  adjustments,  time-phased  replacements,  scheduled  in¬ 
spections,  etc.). 

[  (c)  Maximize  the  extent  to  which  equipment  and  system  per¬ 

formance  can  be  verified,  and  system  calibration  performed 
on  the  end  article  with  minimum  need  for  support  equipment. 


(d)  Design  for  rapid  and  positive  recognition  and  isolation 
of  equipment  malfunction  or  marginal  performance. 

(e)  Design  to  require  the  minimum  personnel  skills  and 
training  needed  to  develop  adequate  maintenance  proficiency. 

(f)  Design  t.c  require  minimum  numbers  and  types  of  facilities, 
support  equipment  (special,  general  and  standard)  required  to 
perform  maintenance. 

(g)  Design  to  require  the  minimum  number  of  parts,  replace¬ 
ment  spares,  and  consumable  maintenance  materials,  by  use  of 
military  standard  items,  standard  cominer  ial  items,  multiple 
use  of  the  same  components  in  the  system,  maximum  use  of 
components  used  in  previous  systems  and  maintenance  of  a  high 
level  of  interchangeability  within  the  system,  and  between 
various  series  or  models  of  the  same  system. 

(h)  Design  to  enhance  and  facilita  e  maximum  field  and 
organizational  self-sufficiency’,  within  the  t  fhnical, 
economic,  and  operational  framework  contained  in  the  Inte¬ 
grated  Maintenance  Management  Plan. 

(i)  oesign  for  the  optimum  accessibility  in  all  systems, 
equipment  and  components  requiring  maintenance,  servicing, 
inspections,  removal  or  replacement. 

(j)  Design  equipment  and  components  which  are  subject  to 
maintenance  to  eliminate  the  possibility  of  improper  in¬ 
stallation. 

(k)  Design  for  maximum  safety  for  both  personnel  and  equip¬ 
ment  involved  in  the  performance  of  maintenance. 

(l)  Wherever  possible  and  logistically  practical,  use  self- 
adjusting,  self-calibrating  and  self-checking  equipment. 

(m)  When  sealed  or  encapsulated  components  are  used  which 
are  subject  to  maintenance,  repair  or  modification,  they 
shall  be  designed,  when  practicable,  to  facilitate  unsealing 
and  resealing  by  maintenance  personnel. 

(3. 2. 2. 6)  Design  Reviews:  Design  reviews  for  maintainability 
requirements  shall  be  accomplished  prior  to  release  of  system 
installation  drawings  and  assembly  production  drawings.  These 
reviews  shall  be  directed  toward  an  analysis  of  troubleshooting 
techniques,  accessibility,  compatibility,  and  adequacy  of  support 


17-4C 


equipment,  human  engineering,  and  life  support  considerations, 
training  requirements,  and  maintenance  support  costs.  For  items 
covered  by  MEAR's  the  contractor  shall  not  release  drawings  for 
initial  fabrication  or  release  purchase  orders  for  procurement 
until  the  maintenance  requirements  and  tasks  have  been  identified 
and  analyzed.  Certification  that  such  review  has  been  made  shall 
be  indicated  by  appropriate  drawing  title  block  signature  and 
date . 

(3.2.4)  Maintainability  Design  Trade-offs.  To  achieve  optimum 
operational  capability,  design  trade-offs  may  be  necessary. 
Maintainability  evaluations  shall  be  made,  as  appropriate,  on  a 
continuing  basis,  as  a  part,  of  system  engineering  studies  to 
establish  support  consequences  of  design  approaches  in  terms  of 
maintenance  resource  requirements  and  development  costs.  Factors 
to  be  considered  in  determining  possible  trade-offs  are  as  fol¬ 
lows:  Criticality  of  failure,  equipment  reliability,  economic 
constraints,  and  performance  requirements. 

(3.3.3)  Personnel  Training  Requirements.  The  contractor  shall 
p.  vide  a  summary  of  training  requirements  in  accordance  with 
WR-25  to  insure  that  military  personnel  will  be  capable  of  main¬ 
taining  the  end  article  and  related  support  equipment.  The 
training  requirements  shall  be  directly  related  to  the  mainten¬ 
ance  requirements  contained  in  the  MEAR’s  and  shall  emphasize 
new  materials,  training  devices  and  techniques  not  currently  in 
use  or  which  are  not  readily  adaptable  to  existing  military 
training  programs. 

(3.4.1)  Publication  Integration.  The  contractor  shall  insure 
that  data  generated  as  the  result  of  the  maintenance  engineering 
analysis  is  appropriately  used  to  provide  the  basis  for  publica¬ 
tions  and  manuals  required  by  publication  specifications  cited 
separately.  It  is  essential  that  the  content  of  technical  man¬ 
uals,  which  are  considered  a  maintenance  resource,  reflect  the 
proper  inter-relationship  of  scheduled  and  unschedule  maintenance 
requirements,  tasks,  support  equipment  and  material  requirements 
and  maintenance  level  capability. 

(3. 5.2.1)  Contractor  Acquired  Spares  and  Repair  Parts  Support 
Material  List?  Preparation  of.  The  contractor  shall  prepare,  on 
the  basi3  of  MEAR's,  support  material  list(s)  for  spares  and 
repair  parts  in  accordance  with  Appendix  B.  These  lists  shall 
include  contractor  and  vendor  items  acquired  to  support  the  end 
article  for  the  duration  of  the  Augmented  Support  Program. 

(3. 5.4.1)  Special  Support  Equipment  Design.  The  contractor  shall 
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immediately  investigate  requirements  for  SSE  as  substantiated  by 
maintenance  engineering  analysis.  Concurrent  with  maintenance 
engineering  analysis,  the  contractor  shall  proceed  with  design 
or  engineering  study  and  shall  prepare  maintenance  engineering 
analysis  on  end  items  of  support  equipment  as  determined  neces¬ 
sary. 


These  procedures  are  applicable  to  contractor  designed  and  fabri¬ 
cated  SSE  as  well  as  SSE  designed  and/or  fabricated  by  vendors 
or  subcontractors.  The  equipment  concerned  is  considered  t^.  be 
of  the  type  necessary  for  service,  maintenance,  test,  repair  or 
overhaul  of  the  end  article  and  systems  or  components  thereof 
and  not  the  type  required  for  developmental  qualification  or 
highly  specialized  technical  laboratory  type  equipment.  The 
Integrated  Maintenance  Management  Team  shall  supply  suoplemental 
maintenance  policy  on  any  item  of  suppo~t  equipment  requested  by 
the  contractor.  Unless  otherwise- specified,  .esign  requirements 
for  special  support  equipment  shall  be  in  accordance  with  the 
specifications  list.ed  in  paragraph  2.1  of  this  document,  except 
that  special  support  equipment  required  solely  for  overhaul  or 
depot  use  may  be  designed  in  accordance  with  best  commercial 
practices . 

(3.7.1)  Control.  Stages;  A  maximum  of  five  stages  for  verifica¬ 
tion  of  maintainability  and  integration  of  maintenance  resources 
are  required  for  control  purposes.  These  stages  are  as  follows: 

(3. 7. 1.1)  Stage  One.  At  the  planning  conference,  the  contrac¬ 
tor  shall  present  data  submitted  during  the  proposal,  updated 
as  appropriate. 

(3. 7. 1.2)  Stage  Two.  Stage  two  shall  be  progressively  imple¬ 
mented  during  breadboarding  or  mock  up  of  the  contract  end 
article,  its  systems  and  equipment,  including  special  support 
equipment.  During  this  stage,  the  contractor  shall  evaluate 
accessibility,  simplicity,  equipment  size,  working  environ¬ 
ment,  maintenance  resource  requirements  and  human  engineering 
considerations.  The  initial  maintainability  predictions  and 
maintenance  resources  requirements  shall  be  verified  and  up¬ 
dated  during  this  stage. 

(j.7.1.3)  Stage  Three.  Stage  three  shall  be  conducted  on 
the  first  representative  production  end  article  which  has 
been  identified  by  the  contractor  and  scheduled  specifically 
for  this  purpose.  During  this  stage  the  maintainability 
program  requirements  shall  be  evaluated  to  insure  tha%>  the 
operational  requirements  can  be  met  without  exceeding  pro- 
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grammed  maintenance  resources.  In  addition,  this  stage  3hall 
include  evaluation  of  compatibility  between  maintenance  re¬ 
sources.  Information  feedback  will  be  initiated  from  observed 
maintenance  action  so  that  eaj.  ly  corrective  action  can  be 
taken  or  initiated. 


(3. 7. 1.4)  Stage  Four.  Stage  four  will  occur  during  trials 
a  which  time  the  achievement  of  the  end  article  maintain¬ 
ability  requirements  will  be  demonstrated.  The  demonstration 
shall  be  performed  on  maintainability  test  aircraft  as  spec¬ 
ified  in  the  test  program.  The  specific  time  phasing  of 
demonstrations  and  proposed  requirements  to  be  demonstrated 
shall  be  stipulated  by  the  contractor  and  shall  be  made  a 
part  of  the  maintainability  program  plan. 


(3. 7. 1.5)  Stage  Five.  Verification  of  the  in-service  and 
-rticle  mai -f ^inability  chai. --Lyrist’  will  bc  nccuuvplioheu 
by  the  Government  in-service  verification  will  be  accomplished 
using  only  these  tools,  equipment,  data,  training,  personnel, 
and  material  resources  which  have  been  programmed  and  provided 
as  a  result  of  the  application  of  this  document. 


3 . 3 . 3  MIL  M  26512C (USAF)  Maintainability  Requirements  for 
Aerospace  Systems  and  Equipment,  U.  S.  Air  Force . 

(1.1)  This  specification  establishe.  the  general  Maintainability 
requirements  for  systems  and  equipmeut  and  provides  maintainabil¬ 
ity  program  policy  and  procedures. 

(3.1)  General  Maintainability  Requirements .  -  The  system/equip- 
ment  maintainability  characteristics  shall  be  such  that  the 
maintenance  required  to  meet  the  planned  mission  can  be  accom¬ 
plished  within  the  limits  specified  in  the  system/equipment 
specification  or  work  i  .atement.  The  maintenance  requirements 
specified  shall  apply  to  all  levels  of  maintenance  in  +he  plan¬ 
ned  maintenance  environment,  and,  depending  upon  the  mission  c r 
the  system/equipment ,  shall  be  stated  in  quantitative  terms  such 
as : 


(a)  Time  (e.g.  mean  and  maximum  down  time,  reaction  time, 
turn  around  time,  mean  and  maximum  times  to  repair,  etc.) 

(b)  Rate  (e.g.  maintenance  manhours/flying  hour,  maintenance 
manhours/specific  maintenance  action,  operational  ready  rate, 
maintenance  hours/operating  hours,  etc.) 


* 


V 


(c)  Maintenance  complexity  (e.g.  number  people  and  skill 


17-51 


levels,  variety  of  AGE,  etc.) 

(d)  Maintenance  costs  (e.g.,  maintenance  costs  per  year, 
etc . ) 

(e)  Accuracy  (e.g.,  tolerances  of  performances). 

(3.2)  Maintainability  Design  Principles.  -  The  design  of  all 
equipment  comprising  the  system  for  which  the  contractor  has 
responsibil4 ty  shall  be  so  developed  that  an  optimum  mix  of 
personnel  skills  and  training,  equipment  complexity,  performance, 
and  reliability  will  be  attained  through.  applic&._!-i.  of  main¬ 
tainability  principles. 

(a)  Design  to  minimize  the  complexity  of  maintenance  tasks. 

(b)  Design  tor  rapid  and  positive  recoqnition  of  equipment 
malfunction  or  marginal  performance. 

(c)  Design  for  rapid  and  positive  identification  of  the  re¬ 
placement  defective  part,  assembly,  or  component. 

(d)  Design  to  require  the  minimum  maintenance  skills  and 
training  needed  to  develop  adequate  maintenance  proficiency. 

(e)  Design  to  require  minimum  numbers  and  types  of  tools  and 
test  equipment  (special  and  standard)  needed  to  perform  main¬ 
tenance  . 

(f)  Design  for  the  optimum  accessibility. 

(g)  Design  for  maximum  safety  for  both  equipment  and  per¬ 
sonnel  involved  in  the  performance  of  maintenance. 

(h)  Maximize  the  extent  that  performance  can  be  verified, 
malfunctions  anticipated  and  located,  and  calibration  per¬ 
formed  . 

(i)  Design  so  the  mean  time  to  accomplish  schedule  and  un¬ 
scheduled  maintenance  is  sufficiently  low  so  as  to  assure 
the  attainment  of  specified  availability  of  the  system/equip¬ 
ment  . 

(j)  Design  to  enhance  and  facilitate  all  levels  of  main¬ 
tenance  action. 

(3.4)  Maintainability  Character ist ics .  -  The  maintainability 


>*- 

if 
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character istics  of  the  equipment  and  components  of  the  system, 
shall  be  determined  or  predicted  in  terms  of  their  contribution 
♦o  the  overall  system  maintainability  characteristics  required 
to  achieve  the  specified  system  requirements  at  each  level  of 
maintenance.  Factors  considered  shall  include  but  not  be  limited 
to  mean- time-between- fai lures ,  mean  time  for  repair,  mean  time 
for  scheduled  maintenance,  operational  requirements,  skills, 
special  equipment,  levels  and  location  of  facilities,  and  mean 
downt ime . 

(3. 5. 3. 2)  Specific  maintainability  program  elements.  -  The 
following  are  the  minimum  program  elements  which  shall  be  in¬ 
corporated  into  the  contractor's  maintainability  program. 

(a)  benign  assistance.  -  Concurrent  with  the  development  of 
the  proposed  program  plan,  the  contractor's  maintainability 
organization  shall  begin  to  provide  design  engineering  with 
maintainability  design  guidelines  and  techniques  for  achieving 
th  e  maintainability  requirement. 

(b)  Design  reviews.  -  Provisions  shall  be  made  to  assure  the 
the  accomplishment  of  design  reviews  at  the  most  appropriate 
»tages  of  system/equipment  acquisition. 

(e)  Review  of  design  changes.  -  Provision  shall  be  made  to 
assure  review  of  all  proposed  design  changes  for  maintain¬ 
ability  quantitative  and  qualitati.e  effects. 

(d)  Corrective  action  system  operation.  -  The  maintainability 
organisation  shall  assure  that  problems  atfecting  the  main¬ 
tainability  of  system, equi pment  shall  be  corrective  action 
responsibility  assigned  and  shall  follow-up  for  timely  resol¬ 
ution  of  such  problems. 

(•)  Maintainability  predictions.  -  Provision  shall  be  made 
for  prediction  of  system  equipment  maintainability  at  selected 
control  points  during  system,  equipment  acquisition. 

(f)  Test  and  demonstration.  -  While  preliminary  testing  and 
demonstration  may,  where  feasible,  be  performed  concurrently 
with  breadboard,  environmental,  ot  other  teste  required  by 
the  design  and  development  program,  provisions  shall  b^  made 
for'  formal  maintainability  t^sts  ; .  n  delivered  article(s). 

(g)  Maintainability  indoctrination  or  training  as  appropriate 
of  contractor  personnel.  -  Provisions  shall  be  made  ror  approp¬ 
riate  maintainability  indoctrination  of  contractor  personnel, 
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such  as,  design  engineering  and  manufacturing  quality  control 
in  the  requirements  and  objectives  of  the  maintainability 
program.  This  indoctr inat ion  may  take  the  form  of  lectures, 
training  films,  workships,  etc. 

(h)  Logistic  support  model.  -  Provisions  shall  be  made  for 
the  Contractor's  maintainability  organization  to  participate 
in  the  development  of  the  logistic  support  model. 

(i)  Maintainability  analysis.  -  Identification  and  manage¬ 
ment  of  the  maintainability  design  and  all  prerequisite 
resources  shall  be  accomplished  through  the  use  of  analysis 
techniques.  A  maintainability  engineering  analysis  of  the 
system  shall  be  accomplished  concurrently  with  the  design 
effort  to  provide  a  systematic  u.  .....tion  ol  the 

tasks  that  will  be  required  in  support  of  the  system/equip¬ 
ment  and  AGE. 

(3.6)  Maintainability  Design  Trade-otfs.  -  To  achieve  optimum 
op^r  at ;  1  canahility,  design  trade-effs  iav  he  ''^ssary.  Th  ' 

major  areas  involved  are  impact  of  equipment  malfunction,  relia¬ 
bility  of  equipment,  economic  limitations,  and  performance 
requirements . 

(4.2)  Maint a  inabi  1  i ty  Demons t  rat  i on .  -  Maintamabi  1  ity  shall  be 
quantitatively  demonstrated  and  evaluate*..  One  method  for 
accomplishment  will  be  found  in  Appendix  A.  The  contractcr  may 
propose  alternate  techniques. 

(4.3)  Maintainability  Records.  -  The  contractor  shall  establish 
and  maintain  records  of  maintainability  information  pertaining  to 
the  contract.  These  records  shall  be  available  for  inspection  by 
the  applicable  procuring  activity  throughout  the  contract. 

3.3.4  MIL-S-23603  (WEP)  System  Readiness/Ma inf  a inab i  1 i ty , 

Avionic  Systems  Design,  General  Specification  for: 

(1.1)  Scope .  -  Tb ' s  document  specifics  one  of  the  major  require¬ 
ments  for  System  Effectiveness  as  it  i elates  to  Avionic  systems 
and  subsystems  (See  6.1.9).  Equipment  complying  with  these 
requirements  shall  be  designed  to  meet  the  requirements  for 
maintainability  and  system  readiness  without  reduction  in  the 
functional  system  performance.  All  levels  of  maintenance  in¬ 
cluding  certain  airborne  maintenance  functions  are  considered 
in  this  specification.  (The  maintainability  terminology 
appearing  in  this  specification  is  defined  in  Far.  6.1.  The 
interrelationship  of  the  terms  is  shown  in  Chart  I). 


(3.3.1)  Operational  Ma in  La  1 nabi 1 ity  Requirements.  -  The  quali¬ 
tative  and  quantitative  characteristics  of  the  equipment  indic¬ 
ated  in  1.2  shall  be  such  that  it  will  be  possible  in  95%  of  all 
the  cases  of  failure,  to  perform  all  corrective  organizational 
maintenance  actions,  other  than  combat  damage,  resulting  from 
two  hours  of  combat  flight  operations  within  a  turn  around 
period  (Max.  4.2.1  (3))  not  exceeding  3  .linutes  .  As  here 
applied  an  organizational  corrective  maintenance  action  includes 
the  following:  (See  6.1  for  definitions  of  terminology). 

(1)  Recognition  of  a  fault. 

(2)  Isolation  of  the  fault  to  a  Weapon  Replaceable  Assembly 

( w  r  s '  t<">  ->  Maintenance  Module  Mother  Board  (See  6.1). 

(31  Repair  of  the  fault. 

(4)  Check  out  of  the  repair. 

(3.3.2)  Maintainability  Indices  for  Organization  Maintenance.  - 

(3  .  3  .  2  .  1 )  Light  Replaceable  Assembly  ( LRA)  Ratio  -  The  rat io 
of  the  number  of  Light  Replaceable  Assemblies  (LRA's)  to  the 
total  numbei  of  Weapon  Replaceable  Assemblies  (WRA's)  (which 
is  equal  to  the  sum  of  the  Hat  d  Replaceable  Assemblies  (HRA' s : 
ulus  the  LRA's)  shall  not  be  less  than  O.ho  unless  otherwise 
specified  in  the  detail  specification.  An  example  showing 
computation  is  shown  in  figures  1  and  1A.  (See  6.1) 

(3.  3.2.2)  N  n- Ambiguity  (N-A)  Ratio  -  The  ratio  of  the  number 
cf  WRA's  fault-isolated  directly  with  built-in  test  features 
and  without  ambiguity  to  the  total  number  of  W'R.V  ■=  shall  not 
be  less  than  O.vS  unless  otherwise  specified,  in  the  detail 
sped  f i cat  ion .  An  example  showing  computation  is  shown  in 
figure  1  and  1A. 

(3.3.2.  3)  Fixed  Interface  (FI)  Ratio  -  The  rati o  o  f  t h e 
number  of  wra's  which  do  not  require  adjustment  or  trimming 
at  installation  in  tne  aircraft  to  the  total  number  cf  wra's 
shall  be  optimized  and  shall  not  be  less  than  1.0  unless 
otherwise  specified  in  the  detail  specification.  An  example 
of  computation  i shown  in  figures  1  and  1  A. 

13.3.3)  Maintainability  Indices  for  Informed l ate  Maintenance.  - 

(3-3.3-13  juick  Replaceable  Assembly  ( wR A )  Ratio  -  The  ratio 
of  the  number  of  yuick  Replaceable  Assemblies  ( QRA )  to  the 
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total  number  of  Shop  Replaceable  Assemblies  (SRA)  (which  is 
equal  to  the  sum  of  ORA's,  the  Bench  Replaceable  Assemblies 
(BRA)  and  the  Inplace  Repairable  Assemblies  (  I  PR  A)  shall  not 
be  less  than  0 . unless  otherwise  specified  in  t  he  dot  =»  i  1 
speci f icat ion.  An  example  >f  com put at  ion  is  shown  in  fig¬ 
ures  2  and  2A.  F;  r  the  definition  of  trie  ter'  ,  SRA,  -.HA, 

BRA  and  I P R A ,  s e e  6.1. 

3  3.3.2)  Shop  N-n-Ambiguity  (SN-A)  R  it.  io  -  The  r at  io  o f  the 
number  of  SRA's  directly  isolated  and  without  ambiguity  to 
the  total  number  of  SRA's  shall  not  be  less  than  0 .u  unless 
otherwise  specified  in  the  detail  speci f icat ion.  An  example 
of  c  imputation  is  shown  in  figures  '  and  2 A. 

(3. 3. 3. 3)  Shop  Fixed  Interface  (SP1)  Rate  -  The  rati-  of 
the  numher  of  SRA's  which  dc  ret  require  ad  rustment  or 
tr  inuni  ng  at  installation  in  a  wra  to  the  total  number  o  f 
SRA's  shall  be  optimized  and  shall  not  be  less  than  1.  1  un¬ 
less  otherwise  specified  in  the  detail  specification.  An 
example  of  computation  is  shown  in  figures  2  and  2a. 

(3.4.2)  Test  Program.  Outline  -  No  later  than  12  1  days  prior  to 
delivery  and  at  least  45  days  prior  to  the  Maintainability  Tests 
the  contractor  shall  s  >bmit  f  ir  review  -id  approval  by  the 
Bureau  of  Naval  Weapons  a  complete  test  program  outline  in  the 
form  of  Part  II  of  MIL-T-183n3 .  The  outline  shall  contain  a 
list  describing  the  tasks  (NC's)  selected  un  in:  4.2.1  to  be 
simulated  during  the  Maintainabi  '  it.  \  Tests.  At  this  time  the 
government  may  add  or  substitute  certain  tasks  for  demons  t.rat  ion. 

(4.1)  Integr  a  ted  Av  i  on  ic  Syst  cm.  Subsystr -m  Readme  ss,-  Mai  rit  a  in¬ 
ability  Testing  _  The  Inteerated  Anionic  System,  Subs  yst  can  shall 
be  tested  by  the  contractor  to  determine  compliance  with  the 
requirements  of  this  specification.  These  tests  shall  be  des¬ 
cribed  in  the  test  procedures  outline  of  3.4.2  as  approved 

the  Bureau  of  Naval  Weapons .  The  tests  shall  be  coordinated 
with  the  test  program  for  the  Integrated  System  when  pract ic- 
able  and  economical  to  do  so.  At  any  rate,  the  tests  shall 
be  conducted  early  enough  so  chat  the  reporting  requirements  of 
paragraph  3.4.3  can  be  met. 

(4.2)  Maintenance  Task  Simulation  -  The  contractor  shall  per¬ 
form  time  studies  of  maintenance  task  simulation  in  a  manner 
representative  af  system  characteristics  in  actual  operation. 

Time  to  accomplish  each  maintenance  task  shall  include  recog¬ 
nition  time,  diagnosis  time,  repair  time  ana  checkout  time.  It 
should  be  recognized  that  active  maintenance  down  time  depends 
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upon  the  ti:ne  required  to  recognize.  Locate,  diagnose,  repair 
and  check  out  the  repair  of  an  equipment  malfunction.  Further¬ 
more,  the  amount  of  required  maintenance  depends  upon  the  equip¬ 
ment  reliability.  Therefore,  in  order  that  the  simulated  main¬ 
tenance  tasks  used  in  the  maintainability  derr  oust  rat  L'-n  will  be 
representative  c-f  normal  operation  all  of  the  above  contributors 
to  active  maintenance  time  shall  be  considered  in  the  t^sk  select¬ 
ion  unless  otherwise  specified.  Selection  of  maintenance  tasks 
shall  be  accomplished  in  accordance  with  4.2.1.  For  the  purpose 
of  maintainability  demonstration,  supply'  down  time  and  waiting 
or  administrative  down  time  shall  be  excluded. 

4 .  S  (JMMARY 


l'n  this  chapter  we  have  outlined  the  need  for  specifications, 
the  complexity  of  the  speci f: nation  tree,  and  the  problems 
incident  to  this  complexity  ana  he  reliability  and  maintain¬ 
ability  technology.  The  three  basic  kinds  of  specifications 
are  described. 

A  selected  list  of  20  specifications,  standards,  and  references 
is  given,  embracing  those  most  commonly  used  to  contract  for 
reliability  and  maintainability.  Detailed  abstracts  from  the 
major  specifications  are  given,  grouping  like  subjects  together 
for  easy  reference. 

ChaDte'-s  22  and  23  contain  recommenced  language  selected  from 
these  specifications  and  modified  or  supplemented  wherever 
improved  practices  are  available.  For  a  .large  program,  such 
language  becomes  part  of  the  Reliability  and  Mi.  ntainability 
Fro  ;ram  Plan.  For  small  programs  the  Chapter  22  and  23  lang¬ 
uage,  or  any  other  in  this  Chapter  17,  may  be  used  directly  as 
appropr ' ate . 


5.  REFERENCES 

(1)  Reliability  of  Mi  lit  iry  Electronic  Equipment  by  the 

Advisory'  Group  on  Reliability  of  Fiectronic  Equipment 
(AGREE),  Office  of  the  Assistant  Secretary  of  Pefense, 
Department  of  Defense,  4  June  1957,  Superintendent  of 

Documents . 


<_o  <.*>  U> 


18- 


1 

1.1 

1.2 

1.3 

2 

3 

3.1 

3.2 

3.3 
3.3.1 

.3.2 
.3.3 
.4 


4 

4.1 

4.2 

4.3 

5 

6 

7 

8 

8.1 

8.2 

8.3 

9 

10 


Chapter  18 
PftRTS  ENGINEERING 

STANDARDS  &  PREFERRED  PARTS 
Standard  Values 
Preferred  Parts 
Preferred  Components 

NON- PREFERRED  PARTS 

TART  SELECTION  AND  APPLICATION 
Known  Reliability  Parts 
Parts  Application 
Derating 

Derating  Factors 
Derating  vs.  Failure  Rate 
General 

Tolerance  Data 

FARTS  OPERATIONAL  DATA 
Design  Data 
Problem  Parts 
Critical  Parts 

PARTS  SPECIFICATION 

ENGINEERING  STOCKROOM 

PARTS  HANDLING 

TRACEABILITY 
Lot  Control 
Serialization 
Control 

PARTS  TESTING 

SUMMARY 


Pa2£ 

18-  3 
18-  3 
18-  5 
18-  7 

18-  V 

18-  7 
18-  8 
18-  9 
18-10 
18-10 
18-13 
18-20 
18-20 

18-21 

18-21 

18-28 

18-28 

18-29 

18-30 

18-31 

18-31 

18-32 

18-32 

18-33 

18-3  3 

18-34 


11 


REFERENCES 


18  3  4 


18-2 


Chapter  18 
PARTS  ENGINEERING 

Probably  no  other  contractor  activity  has  a  greater  impact  on 
system  or  component  reliability  than  his  attention  to  the  relia¬ 
bility  of  the  parts  he  selects  and  uses.  This  is  true  because, 
in  a  typical  design,  failure  of  any  one  of  most  parts  causes 
system  failure.  These  are  called  "critical"  parts,  as  opposed 
to  other  parts  whose  failure  would  not  cause  system  failure. 
Failure  of  a  part  is  its  departure  from  the  functional  specifi¬ 
cations  upon  which  its  application  depends,  including  degradation 
or  cessation  of  function. 

But  it  does  not  follow  that  if  all  parts  have  perfect  reliabil¬ 
ity,  the  system  will  have  perfect  reliability.  This  is  so  be¬ 
cause  the  application  of  parts  and  their  interfaces,  such  as 
tolerance  drift  compatibility,  can  be  such  that  the  system  will 
fail  to  function  properly  even  with  perfectly  reliable  parts. 

Nor  dees  it  follow,  as  one  often  hears,  that  parts  improvement  is 
the  only  way  to  achieve  required  reliability.  One  other  way  is 
the  use  of  judicious  redundancy,  which  protects  the  system  from 
failure  of  certain  parts. 

So  we  see  that  parts  control  is  an  extremely  necessary,  but  not 
sufficient,  task  achieve  the  required  equipment  or  system  re¬ 
liability. 

Many  contractors  have  excellent  parts  control  procedures,  but 
many  do  not.  The  basic  problem  is  to  prevent  the  selection  of 
parts  about  which  there  is  inadequate  knowledge,  or  their  use 
in  a  way  that  degrades  reliability.  It  is  particularly  serious 
in  the  electronics  or  mechanisms  areas,  where  the  green  young 
engineer  can  become  sold  on  an  elegant  new  part  in  the  vendors 
catalog.  It  is  "just  what  he  needs"  but  has  no  history,  no  ped¬ 
igree,  no  MTBF  rating. 

For  the  BuShips  engineer,  this  chapter  presents  a  picture  of  the 
contractor  work  commonly  called  "parts  engineering",  with  atten¬ 
tion  to  activities  directly  affecting  reliability.  Since  there 
are  many  such  groups  that  do  not  yet  control  reliability  in  the 
manner  to  be  outlined,  the  Bureau  engineer  should  look  for  evi¬ 
dence  that  each  of  these  activities  is  effectively  handled  by 
at  least  some  part  of  the  contractors  organization.  They  can  be 
evaluated  as  discussed  in  chapter  23  section  4.2. 
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1.  STANDARDS  &  PREFERRED  PARTS 

Many  design  engineers  have  resisted  standardization,  on  the 
ground  that  it  restricts  their  freedom  for  exercise  of  unbridled 
creativity  and  "progress"  to  new  things.  Now  that  we  are  bending 
the  objective  from  “new  things"  to  "thing  that  keep  working" , 
such  resistance  amounts  to  poor  engineering. 

But  unless  such  standards  are  kept  vigilantly  up  to  date  with 
advancing  state  of  the  art,  they  can  discourage  initiative  for 
new  developments.  They  mi  t  be  constantly  reviewed  to  add  new 
standards.  In  the  case  of  new  physical  hardware  standards,  very 
thorough  reliability  verification  must  precede  their  establish¬ 
ment  prior  to  withdrawal  (for  new  design)  of  obsolete  standards. 

The  American  Standards  Association  (1)  has  been  established  for 
national  approval  of  standards  sources  and  for  distribution  of 
many  standards.  Now  let's  review  those  pertinent  to  our  needs. 

1.1  STANDARD  SIZES  &  VALUES 

Reliability  is  improved  by  experience.  To  the  extent  any  manu¬ 
facturer  can  produce  larger  quantities  of  fewer  sizes,  he  learns 
more  about  each  size  by  feedback  from  more  users,  and  improves 
his  design. 

Many  years  ago  the  old  Radio  Manufacturers  Association  (now 
Electronic  Industries  Association)  saw  the  great  economic  and 
reliability  advantage  of  establishing  stock  values.  They  chose 
a  set  of  values  in  which  each  successive  value  is  24v/l0  greater 
than  the  preceding,  for  t  5%  tolerance.  Then  for  110%  tolerance 
every  other  value  is  standard,  and  for  120%  every  fourth  value 
is  standard.  This  system  achieved  complete  coverage  of  all  values, 
since  adjacent  high  and  low  tolerances  nearly  coincide.  In 
Figure  18-4  these  values  are  shown  in  the  "Choice  1,  2  and  3" 
columns,  and  each  value  can  be  multiplied  or  divided  by  10  or  any 
multiple  of  10. 

The  advent  of  film-type  (carbon  and  metal  film)  with  much  better 
control  of  resistance  values  prompted  the  military  to  extend  the 
system  to  tl%  values,  as  shown  in  the  Military  Standard  MS  90169 
"Choice  3  and  4"  columns.  Nearly  all  resistor  and  capacitor 
manufacturers  now  furnish  all  of  these  standard  values. 

There  are  many  electrical  and  electronic  circuit  applications 
where  t20%  resistance  or  capacitance  values  are  more  than  adequate. 
So  in  the  interest  of  reliability  and  economy  the  design  engineer 
should  always  specify  the  "Choice  1"  values  when  they  will  serve 
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FILM  RESISTOR  STANDARD  VALUES 
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the  purpose,  and  if  not,  use  Choice  2,  etc. 

In  many  cases  the  design  engineer  uses  precision  (film-type)  re¬ 
sistors  where  *.he  larger  drift  of  composition  or  other  resistors 
cannot  be  tolerated.  But  to  take  advantage  of  standard  part  re¬ 
liability  and  economy,  he  must  use  these  specified  values  even  if 
the  precision  (nominal  value)  is  not  needed.  Many  companies  order 
only  il%  film- type  resistors,  to  reduce  the  problem  of  calculating 
circuit  performance  via  tolerance  evaluation. 

While  the  above  system  is  well-established  for  small  resistors 
and  capacitors,  its  advantages  apply  equally  wel)  for  all  kinds 
of  parts  that  are  used  in  quantity.  Screw  sizes  and  threads, 
and  pipe  have  used  similar  systems  for  a  hundred  years.  As  our 
technology  develops  new  kinds  of  parts,  the  design  engineer  can 
contribute  to  reliability  by  specifying  values  from  such  systems. 

1.2  PREFERRED  PARTS 


Here  a  semantic  problem  has  arisen.  Several  military  and  NASA 
agencies  have  established  "standard"  parts  lists,  such  as  MIL 
STD  242E.  The  criteria  for  inclusion  in  such  lic^c  varies  widely, 
from  mere  volume  of  use  to  fairly  rigid  life  test  qualification. 
Similarly  many  contractors  have  established  such  "standard  parts" 
lists,  also  with  widely-var iant  criteria.  Thus  the  term  "stan¬ 
dard  part"  does  not  mean  the  "best  part",  or  parts,  to  use  among 
alternatives.  So  the  term  "preferred  part"  has  emerged,  to  con¬ 
note  a  much  higher  degree  of  selectivity.  A  fairly  high  propor¬ 
tion  of  "standard"  parts  are  not  eligible  for  "preferred  part" 
status  with  contractors  working  to  specified  reliability.  Let's 
review  the  advantages  of  the  establishment  of  a  truly  minimum 
number  of  preferred  parts: 

Manufacturers  Cost  Reduction:  Obviously  the  rore  a  manufac¬ 
turer  can  make  of  fewer  types,  the  lower  his  unit  production 
costs  will  be.  Moreover,  as  competitors  are  attracted  by  the 
volume,  multiple  sources  are  assured  and  competitive  process 
refinements  drive  the  price  still  lower  and  keep  it  down. 

Reliability:  As  higher  quantities  of  fewer  types  are  pro¬ 
duced,  the  manufacturer  can  generally  afford  more  specialized 
test  facilities,  and  engineering  refinement,  spreading  their 
cost  over  more  units.  He  learns  more  about  the  failure  modes 
of  each  type  from  his  own  experience  and  from  his  customers, 
and  thus  can  take  more  remedial  action  to  raise  reliability. 
Since  he  tests  more  of  them,  his  confidence  level  for  a  given 
failure  rate  is  higher,  or  conversely  the  failure  rate  is 
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lower  for  conventional  confidence  levels. 

Contractors  Cos,.  Reduction;  Since  the  part  manufacturer  can 
afford  better  testing,  and  is  usually  better  able  to  do  it 
than  his  customer,  standardization  generally  reduces  the  con¬ 
tractors  own  verification  or  acceptance  testing.  But  the 
sheer  reduction  of  paperwork  for  multiple  specifications, 
procurement  requisitions,  purchase  orders,  and  liaison  con¬ 
stitutes  substantial  cost  reduction.  Parts  lists  are  simpli¬ 
fied.  Obviously  the  contractors  parts  inventory  investment 
and  control  cost  is  greatly  reduced. 

Ownership  Cost;  As  fewer  types  of  parts  are  required  for 
maintenance,  with  higher  quantities  of  each,  the  logistics 
cost  is  reduced.  Fewer  parts  have  to  be  stocked  on  board, 
at  shipyards,  at  supply  centers,  and  at  the  manufacturer's 
plant.  Fewer  parts  are  needed  in  the  "pipe  line".  Fewer 
parts  permits  better  handling  at  lower  cost.  But  also  the 
higher  reliability  of  preferred  parts  reduces  maintenance  and 
therefore  logistic  costs. 

Operational  Data:  Higher  quantities  of  fewer  types  permit 
the  acquisition  of  more  complete  operational  information  on 
each  part,  for  feedback  to  design  engineers.  And  the  data 
has  better  precision.  Tha  design  engineer  has  a  more  real¬ 
istic  basis  for  decision. 

Tolerance  Limits;  Part  standardization  should  particularly 
cover  configuration,  such  as  shape,  lead  length  and  material, 
mounting  dimensions,  etc.  Here  the  Bureau  or  the  contractor 
can  usually  specify  limits  in  such  a  way  as  to  encompass 
several  suppliers  standard  products  to  preserve  competition, 
economy,  and  availability  for  delivery.  To  standardize  on 
one  suppliers  exact  design  is  sometimes  necessary,  but  rarely 
desirable.  Without  competition,  or  with  patent  advantage  over 
competition,  there  is  not  much  incentive  for  reliability  and 
cost  improvement. 

With  the  above  advantages  in  mind,  many  contractors  have  estab¬ 
lished  policies  and  procedures  that  require  the  design  engineer 
to  (a)  select  and  specify  a  preferred  part,  or  (b)  convince  a 
parts  engineering  specialist  t’.sat  no  existing  preferred  part 
will  serve  the  purpose.  As  repeated  demands  for  a  kind  of  part 
occur,  this  procedure  serves  to  initiate  the  establishment  of  a 
new  preferred  part,  and  possible  removal  of  a  superseded  part 
from  the  preferred  parts  list.  The  Bureau  engineer  can  look  for 
and  encourage  such  contractor  procedures. 
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1.3  PREFERRED  COMPONENTS 

Components  such  as  regulator  valves  or  amplifier  circuits,  made 
up  of  parts,  can  be  either  selected  from  available  supplier  pro¬ 
ducts  or  developed  by  the  design  engineer,  for  wide  use  across 
a  range  of  higher-level  designs. 

The  above  considerations  for  parts  apply  equally  well  to  compon¬ 
ents,  though  there  are  fewer  such  standards  and  they  are  more 
complex.  Many  contractors  maintain  a  file  of  thoroughly-proven 
circuits,  which  may  be  either  used  directly  or  modified  to  avoid 
the  unreliability  and  cost  of  complete  reinvention.  But,  sadly, 
the  amount  of  such  reinvention  in  the  U.  S.  must  be  staggering, 
for  sheer  lack  of  communication. 


2.  NON-PREFERRED  PARTS 

As  discussed  above,  some  government  agencies  have  established 
"standard  parts"  or  "qualified  parts"  lists  of  parts  qualified 
to  some  criteria  appropriate  at  the  time.  Such  agencies  then 
take  the  posit-ion  tha;'  the  contracto»"  n"’st  oht-»ir>  t^eir  approval 
to  use  each  part  not  on  the  list,  as  in  MIL  E  16400E  section 
3.4.1.  The  contractor  must  give  his  justification,  which  is 
given  on  grounds  of  performance  capability,  reliability,  cost, 
etc.  Such  negotiation  is  usually  conducted  with  the  agency  by 
the  contractors  parts  engineering  specialists,  who  are  most 
likely  to  have  wide  knowledge  oc  available  part  capabilities. 

But  in  order  to  achieve  required  reliability,  and  whether  or  not 
the  contractors  customer  has  a  standard  parts  list,  the  same  kind 
of  control  must  be  exercised  over  non-preferred  parts.  That  is, 
if  the  contractor  has  established  a  good  preferred  parts  list, 
on  which  only  parts  of  known  history  and  reliability  appear,  there 
must  be  a  procedure  that  prevents  use  of  other  parts  until  the 
pertinent  preferred  parts  are  adequately  considered.  This  is 
normally  handled  by  requiring  a  parts  engineering  or  reliability 
specialist  approval  of  drawings  prior  to  design  review  for  final 
release.  Some  contractors  require  such  approval  of  all  parts 
procurement  requisitions,  which  is  pretty  late  and  requires  the 
parts  engineering  people  to  go  back  over  the  drawings  and  anal¬ 
yses  anyway. 


3 .  PART  SELECTION  AND  APPLICATION 

Two  major  causes  of  unreliability  are  (a)  selection  of  a  part 
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without  really  knowing  its  reliability  history,  and  (b)  improperly 
using  the  part  in  design,  so  that  its  capabilities  are  exceeded. 
Preferred  Parts  lists  establish  a  "bank"  of  known-reliability 
parts,  but  they  cannot  contain  all  parts  needed  for  a  new  design. 

3.1  KNOWN  RELIABILITY  PARTS 


The  achievement  of  specified  component  reliability  begins  with 
the  selection  of  parts  of  known  reliability.  For  most  situations 
the  component  failure  rate  is  the  simple  sum  of  the  failure  rates 
of  its  parts  whose  failure  would  cause  component  failure.  This  is 
true  whether  or  not  the  actual  part  failure  rate  values  are  known. 

In  general  the  only  sources  of  dependable  information  on  a  speci¬ 
fic  part  are  the  supplier  of  the  part  and  the  users  of  the  part, 
if  they  have  kept  records  of  failures  vs.  stress  time.  And  some 
suppliers  are  reluctant  to  disclose  such  information  for  fear 
they  will  fare  unfavorably  relative  to  competition.  Some  con¬ 
tractors  or  subcontractors  who  use  the  part  may  keen  y->od  records  , 
and  may  provide  such  specific  information. 

One  way  to  get  ^a-tc  of  known  reliability  is  to  specify  the  re¬ 
quired  quantitative  value  and  the  verification  requirement  to 
parts  suppliers,  and  ask  for  quotations.  Some  wul  decline  to 
quote,  indicating  they  either  do  not  know  how  to  get  the  required 
value,  or  are  unwilling  to  guarantee  their  assertions.  Others 
will  quote  without  understanding  the  problem,  then  later  renege 
when  they  realize  what  is  required.  But  many  today  do  understand, 
do  keep  test  and  operatic  lal  records,  and  can  be  relied  upon  to 
show'  convincingly  what  value  they  can  achieve.  And  most  of  these 
are  willing  to  guarantee  the  value. 

The  commonest  quantification  of  parts  reliability  is  in  terms  of 
maximum  failure  rate,  or  failures  per  million  hours  of  stress 
(or,  somewhat  confusingly,  in  X  failures/1000  hours).  But  there 
can  b°  situations  where  stress  time  records  are  impractical,  so 
that  maximum  unreliability  must  be  specified  as  the  probability 
of  failure  in  a  given  production  lot  or  lots,  for  specified  en¬ 
vironment  and  stress.  This  is  roughly  equivalent  to  a  fraction 
of  the  overlap  area  between  stress  and  strength  distribution 
curves . 

"Established-Reliability  Farts"  specifications  MIL  R  38100  (4) 
are  being  developed  by  DOD  under  Air  Force  sponsorship,  as  an 
out-grow^th  of  the  Minuteman  high-reliability  electronic  parts 
program.  These  specifications  call  for  very  tight  design  and 
manufacturing  controls,  special  handling,  and  a  continuous  test 
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program  to  verify  the  achieved  reliability.  As  a  result,  a  10- 
to-1  failure  rate  reduction  was  achieved  for  about  a  dozen  common 
electronic  parts,  which  can  be  procured  from  their  manu  fiicturers  .' 
The  program!  is  expected  to  be  expanded  to  other  electronic  and 
mechanical  rarts.  In  the  meantime  some  contractors  are  using  the 
MIL-R-38100  content  as  a  model  to  develop  their  own  specification 
to  vendors,  and  the  BuShips  engineers  can  c*o  the  same  for  critical 
requirements  but  redundancy  of  specifications  should  be  discouraged. 

Today,  however,  the  quantitative  reliability  records  for  most 
specific  parts  (as  opposed  to  part  classes)  do  not  yet  exist. 

The  design  engineer  must  then  look  for  other  assurance  of  "known 
reliability".  For  example  if  he  knows  that  hundreds  or  even 
dozens  of  a  certain  part  have  been  used  (stressed)  for  years, 
and  he  can  find  no  evidence  that  it  has  ever  failed  (except  by 
misuse) ,  he  has  good  intuitive  cpnfidence  that  it  has  high  relia¬ 
bility.  On  the  other  hand  he  must  systematically  and  thoroughly 
look  for  such  evidence  of  failure  before  real  confidence  can 
develop.  The  Bureau  should  require  the  contractor  to  seek  such 
evidence . 

3.2 _ PARTS  APPLICATION 

Improper  use  of  parts  utterly  wastes  the  time  ana  cost  of  careful 
known-reliability  part  selection.  The  selection  and  application 
of  parts  of  course  consrti  futes  a  large  part  of  the  design  process. 

It  is  the  design  engineer's  job  to  balance  dozens  of  consider¬ 
ations  for  each  decision. 

But  *K>st  design  engineers  are  not  experts  on  more  than  a  few 
parts  with  which  their  experience  is  extensive.  Almost  every 
new  design  involves  parts  with  which  the  design  engineer  is 
only  barely  acquainted.  To  solve  this  problem  most  we 11 -organized 
contractors  have  long-established  groups  of  parts  engineering 
specialists ,  each  of  whom  work  only  with  a  few  kinds  of  parts. 

Ovsr  a  period  of  ti^e  they  know  more  about  the  capabilities  and 
1 tnitat ions  of  some  specific  parts  than  anyone  in  the  contractors' 
organization.  The  great  bulk  of  this  knowledge  is  perishable, 
not  on  spec  sheets,  not  in  handbooks. 

However  it  ia  imperative  that  the  parts  specialists  know  the 
reliability  of  each  part,  and  how  such  reliability  is  affected 
by  application.  Many  well-established  Parts  Engineering  groups 
do  not  have  this  knowledge. 

Hardly  anything  will  contribute  more  to  reliability  than  the 
detailed  review  of  a  proposed  design  by  the  cognizant  parts 
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specialists.  When  they  are  inclined  to  recommend  against  use  of 
a  specific  part  in  a  certain  way,  or  to  recommend  an  alternative 
part  or  manner  of  its  use,  there  should  be  thorough  discussion 
before  the  design  engineers  decision  is  made.  Some  contractors 
have  procedures  whereby  only  the  top  Engineering  Manager  can 
override  a  parts  engineer's  disapproval  of  a  part  application. 
Knowledge  of  this  causes  the  design  engineer  to  be  very  sure 
of  tt 1 8  ground  before  taking  a  position.  The  Bureau  engineer 
should  make  sure  some  such  control  is  in  effect. 

3.3  DERATING 


Intuitively  every  design  engineer  feels  that  reliability  is 
improved  by  using  parts  rated  much  higher  than  the  expected 
stress.  That  is,  he  "derates''  the  parts  for  his  application. 

It  is  equivalent  to  increasing  the  "safety  factor".  Unfort¬ 
unately  this  practice  also  increases  cost,  weight,  and  volume. 

If  operational  experience  shows  no  failures,  he  never  knows  how 
much,  if  any,  unnecessary  cost,  weight  and  volume  he  nas  in¬ 
curred.  We  are  all  aware  of  such  examples  of  "overdesign". 
Nevertheless,  judicious  derating  is  a  powerful  aid  to  reli  - 
bility.  There  are  two  basic  approaches: 

3.3.1  Derating  Factors :  In  the  absence  of  good  failure  rate 
data,  the  parts  engineering  specialists  may  establish  quite 
arbitrary  derating  factors  for  each  kind  of  part.  These  are 
based  on  long  experience,  trial  and  error,  and  [judgment.  They 
have  been  quite  successful,  anu  are  very  widely  used,  but  un¬ 
doubtedly  cause  some  degree  of  “overdesign".  Examples  of  these 
are  shown  in  Figure  18-11,  used  bY  a  major  con¬ 

tractor,  and  18-12  from  MIL  HDBK  217.  Policies  and  procedures 
are  established  whereby  all  design  engineers  are  required  to  use 
the  indicated  minimum  deratings. 

Manufacturers’  catalogs  are  somewhat  confusing  regarding  resistor 
power  rating  since  they  often  give  three  ratings.  The  most 
optimistic  rating  is  the  manufacturer's  (commercial),  while  the 
most  pessimistic  is  the  MIL-R-93A.  The  MIL-R-9444  specification 
permits  more  power  to  be  dissipated  in  a  given  size  resistor 
than  M IL-R-93 A  permits,  yet  it  contemplates  a  more  severe  environ¬ 
ment.  This  means  that  resistors  of  the  encapsulated  variety  which 
meet  both  of  these  military  specifications  are  much  better  than 
the  MIL-R-93A  requirements  and  we  needlessly  penalize  ourselves 
when  we  derate  to  these  pessimistic  watt  values. 

Nearly  all  transistors  are  rated  by  the  manufacturer  on  the  basis 
of  an  absolute  max  ^um  system.  These  ratings  are  not  conservative 
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and,  if  ex:-.  _,®d,  will  result  in  immediate  failure  or  drastically 
reduced  life  expectancy  of  the  device.  Figure  18-11 ,  previously 
called  out,  extracts  the  derating  values  from  a  quite  thorough 
exposition  (5)  on  good  transistor  circuit  design.  The  following 
consolidates  the  best  derating  practices  used  by  experience  design 
engineers . 

It  has  been  estimated  that  95  per  cent  of  all  transistor  failures 
are  due  to  voltage  breakdown.  Although  this  estimate  may  be  some¬ 
what  high,  it  emphasizes  the  necessity  to  derate  the  voltages 
eDplied  to  transistors  to  as  low  a  value  as  possible  consistent 
with  required  performance.  Some  research  work  indicates  that  the 
reliability  of  germanium  transistors  can  be  increased  by  a  factor 
of  10  by  operating  at  half  the  rated  voltage. 

Common  practice  is  to  limit  the  peak  collector  voltage,  including 
a-c  swing  ana  surge  voltages,  to  7 5  per  cent  maximum  rated  value 
BVcbo  or  BVceq*  Since  seme  typ^s  of  voltage  breakdown  occur  at 
lower  voltages  as  the  ambient  temperature  is  reduced,  the  delating 
factor  of  7.  per  cent  should  be  applied  to  the  breakdown  voltage 
at  the  lowest  temperature  of  interest.  Avalanche  breakdown  margin 
should  also  be  checked  at  the  maximum  expected  temperature. 

Junction  temperature  is  an  important  factor  in  transistor  relia¬ 
bility.  Maximum  junction  temperatures  of  15C°C  for  silicon  and 
8 5° C  for  germanium  are  considered  safe.  However,  reliability 
is  greatly  improved  if  junction  temperatures  are  kept  as  low  as 
possible.  Some  researchers  indicate  the  re’iability  doubles  for 
each  10°c  reduction  in  junction  temperature.  Normally,  junction 
temperature  is  determined  by  the  power  dissipated  in  the  tran¬ 
sistor  due  to  collector  current  and  voltage.  However,  some 
applications  (e.g.  dc  to  dc  converters)  will  have  a  substantial 
amount  of  power  dissipated  in  the  base  circuit,  this  power  must 
be  considered  when  predicting  junction  temperatures. 

3.3.2  Derating  vs.  Failure  Rate;  When  good  failure  rate  data 
is  available,  a  more  rational  approach  may  be  used.  The  design 
engineer  selects  the  parts  he  would  like  to  use,  designs  the  cir¬ 
cuit  or  mechanical  assembly,  and  calculates  (usually  simple  fail¬ 
ure  rate  addition)  the  total  failure  rate  of  his  design.  If  it 
is  too  high  or  borderline,  ho  then  refers  to  failure-rate-vs- 
s tress  curves,  snd  derates  judiciously  until  the  total  is  low 
enough.  Examples  of  such  curves  are  shown  in  Figures  18-14  through 
18-15  -  18-19  from  MIL-Hdbk-2 17 .  These  are  " generic1'  curves, 

however,  useful  only  for  relative  failure  rate  comparison.  They 
do  not  express  absolute  values  for  the  specific  papt  selected. 


ELECTRON  TUBE  DERATING 


Voltage  &  Current  (except  heater) 
Power  (all  elements)  . . 


MINIATURE  TUBE  FAILURE  RATE 


Bulb  temperature,  *C  (maximum  rating. 


FILM  RESISTOR  FAILURE  RATE 


H8  8  % 


sjmoq  0O0l/%  3-mipjj 


.02 


TRANSFORMER  FAILURE  RATE 


18-20 


Parts  engineering  specialists  ar°  responsible  for  keeping  up-to- 
date  such  data  in  the  hands  of  the  design  engineers,  particularly 
as  data  is  collected  from  actual  test  and  operational  experience 
with  the  specific  parts,  and  for  approving  part  application  cn 
the  basis  of  such  derating. 

3.3.3  General :  In  the  electronics  industry,  data  (2,  3)  on 
failure  rates  vs.  stress  is  -"'ailabie  for  a  number  of  common 
parts.  The  curves  can  be  used  to  get  at  least  a  rough  idea  of 
the  reliability  improvement  available  through  derating  by  any 
amount . 

In  the  mechanical  and  structural  fields  such  data  is  sometimes 
obtainable  from  the  manufacturer  or  users,  but  time  rate  data  is 
not  available  yet  in  handbook  form.  In  using  the  manufacturer's 
rating  and  single  design  stress  values,  i.he  design  engineer  has 
to  keep  in  mind  that  they  are  really  distributions,  not  single 
values.  He  has  to  find  out  either  the  worst-case  "tolerances" 
for  both  stress  and  strength,  or  preferably  plot  the  distributions 
themselves . 

The  Bureau  engineer  should  make  very  sure  that  some  such  national 
derating  approach  is  established  and  enforced,  preferably  by 
cognizant  parts  engineering  specialists. 

3.4  TOLERANCE  DATA 


In  quantity  manufacture,  all  parts  characteristics  have  statis¬ 
tical  distributions.  That  is,  any  one  characteristic  (such  as 
length  or  resistance)  has  a  nominal  or  mean  value,  and  a  variance 
above  and  below  it.  We  call  the  extreme  values  of  <_he  variance 
"Tolerances".  These  distributions  are  basically  affected  by 
manufacturing  lot,  and  by  techniques  for  selection  of  close-toler¬ 
ance  parts  out  or  wide-tolerance  lots. 

In  addition  to  such  manufacturing  variance  chore  is  application 
variance  regardless  of  quantity.  That  is,  there  are  distribu¬ 
tions  of  each  characteristic  resulting  from  environment  (tem¬ 
perature,  etc.)  stress  (pressure,  voltage,  etc.),  and  time 
(cold  flow,  drift,  aging,  etc.).  Such  distributions  or  toler¬ 
ances  must  be  added  to  the  manufacturing  distributions  or 
tolerances  in  ore  'r  io  determine  the  real  operational  distribu¬ 
tion.  An  example  is  steam  line  design  whore  0-rings  arc  used 
to  absorb  tolerances. 

A  design  is  never  complete  until  the  design  engineer  has  made 
sure  that  the  distributions  cr  tolerances  cannot  combine  in 


18-21 


such  a  way  as  to  interfere  with  the  intended  function.  The 
Bureau  engineer  must  make  sure  that  adequate  suu.  analysis  is 
conducted . 

In  a  complex  circuit,  mechanism,  or  structure  it  is  necessary  to 
consider  the  overa' 1  effect  of  the  expected  range  of  manufactur¬ 
ing  variance,  operational  environment  and  all  stresses,  and  the 
effect  of  time.  Chapter  13,  section  2.5  outlines  Worst-Case 
Tolerance  Analysis,  Statistical  Tolerance  Analysis,  and  Marginal 
Checking,  all  used  for  this  purpose.  All  such  tolerance  eval¬ 
uation  depends  on  some  depth  of  part  tolerance  data,  and  the 
parts  engineering  specialist  again  is  in  the  best  position  to 
develop,  publish,  and  update  such  data  for  broad  use  for  by 
design  engineers.  Figures  16^22  through  18-26  show  some  excellent 
data  (5)  for  this  purpose,  not  available  in  the  MIL  handbooks. 

4.  PARTS  OPERATIONAL  DATA 


4.1  DESIGN  DATA 


In  order  to  design  to  meet  a  specified  reliability,  the  design 
engineer  must  obtain  some  idea  of  the  reliability  of  the  parts 
he  tentatively  proposes  to  use.  Such  information  is  often 
meager  indeed.  While  some  electronic  parts  data  is  prolific,  it 
is  also  contradictory  and  therefore  generates  little  confidence. 

So-called  "generic"  failure  rate  data  was  developed  initially 
by  RCA  for  electronic  parts,  and  subsequently  refined  by  several 
contractors  and  government  agencies.  It  is  currently  available 
in  MIL-Hdbk-217  (2)  and  the  Farada  (3)  books,  and  in  NAVSHIPS 
93820.  But  this  data  is  subject  to  wide  variance,  and  useful 
only  for  preliminary  or  comparative  analysis.  It  tells  little 
about  the  failure  rate  of  a  speci f ic  suppliers  part  that  the 
design  engineer  may  consider  using. 

Very  few  contractors  have  developed  continuous ly-operat ing  data- 
reduction  systems  that  provide  information  to  design  engineers 
in  convenient  form.  Figure  18-27  shows  a  rather  comprehensive 
system  now  under  development  by  a  major  contractor.  A  "Data 
Integration"  group  sets  up  an  automatic  "all  source"  flow  of 
pertinent  data  from  industry  (217,  Farada,  Etc.,  above),  supp¬ 
liers,  design  evaluation  tests,  manufacturing  teats,  contractor 
and  possibly  BuShips  tests  and  Navy  operations.  Using  controlled 
formats,  the  group  screens  the  data  to  weed  out  the  non-signifi¬ 
cant,  interrogates  sources  where  feasible  for  clar i f ication , 
conduces  running  analyses  for  current  estimates,  and  reduces  the 
result  to  an  updated  punch  card.  The  updated  cards  accumulate, 


RESISTANCE  VALUE  %  CHANGES 


•This  column  also  applies  to  w:re-wound  resistors,  except  for  manufacturing  tolerance. 

IOTE:  Derating  is  with  reference  to  the  "application”  rating,  or  allowable  load  at  given  conditions. 
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of  the  uncertainty  of  value.  For  these  parts  in  particular,  the  manufacturer's  tolerance  value,  if  different  from 
that  assumed  above,  should  be  used  in  recalculating  applicable  limits 
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displace  prior  cards  in  sorting,  and  regular  monthly  printouts 
are  produced. 

The  design  engineering  printout  across  the  bottom  of  the  figure 
(a)  identifies  each  part  or  component  for  which  data  is  available, 
shew*  (b)  the  environment  (such  as  ground  air-conditioned  environ¬ 
ment,  GAC)  and  stress  (such  as  15  kpsi)  for  which  the  data  applies, 
(c)  the  current  failure  rate  estimate,  distribution  (such  a3 
Weibull  parameter  code),  and  life  (such  as  1000  hours  to  rising 
failure  rate) ,  (d)  the  MTTR,  preventive  maintenance  fraction  of 
stress  time,  and  maintenance  cost  per  1000  hours,  and  (e)  the 
approximate  component  cost,  delivery,  and  weight. 

With  regular  printouts  of  this  nature,  the  design  engineer  is  in 
a  position  to  compare  one  part  with  another  on  a  truly  balanced 
and  integrated  basis,  relying  less  on  intuition. 

The  Bureau  engineer,  in  evaluating  competitive  contractors,  will 
know  that  the  contractor  who  has  such  a  system  is  making  the  most 
of  data  feedback  to  design,  and  that  is  equivalent  to  high-relia¬ 
bility  design  the  first  time. 

4.2  PROBLEM  PARTS 


Part*  fail  for  many  reasons.  Three  of  them  are  (a)  the  part  is 
bad,  or  does  not  meet  its  own  specifications,  (b)  its  specifi¬ 
cations  are  wrong,  and  (c)  it  is  improperly  used,  whether  or  not 
correctly  specified.  Part  of  the  failure  diagnosis  job  (Chapter 
16)  is  to  find  out  which  for  sure.  In  any  case  a  running 
corrective  action  log  should  be  kept  (Chapter  21,  section  8) 
until  the  problem  is  completely  resolved. 

If  the  verdict  is  reason  (a)  above,  the  part  should  be  removed 
from  the  Design  Data  (Section  4.1  above)  printout,  or  at  least 
flagged  thereon,  until  the  problem  is  resolved,  If  it  is  reason 
(c)  it  should  not  be  removed  because  it  is  not  a  defective  part. 
The  Bureau  engineer  should  see  that  such  procedures  are  used. 

4.3  CRITICAL  PARTS 


A  critical  part  may  be  defined  as  one  whose  failure  would  cause 
system  failure.  Note  that  in  this  case  it  is  not  a  question  of 
the  reliability  of  the  part  itself,  but  rather  only  of  how  it  is 
u~ed.  A  preferred  part  of  "perfect"  reliability  can  be,  and 
should  be,  a  critical  part.  Unhappily  many  people  persist  in 
thinking  of  critical  parts  as  undesirable,  and  there  are  in¬ 
structions  to  "eliminate"  them.  This  is  nonsense,  because  what- 
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ever  replaces  them  then  becomes  critical. 

There  are  several  speci f ications  that  require  "special  handling" 
(see  section  7  below)  for  critical  parts,  for  obvious  reasons. 
Thus  "critical  parts  lists"  are  required  of  the  contractor.  But 
since  they  only  reflect  how  the  part  is  used,  and  should  not  be 
affected  by  its  inherent  reliability,  a  given  part  will  be 
critical  in  some  systems  and  not  in  others. 

"Criticality",  on  the  other  hand,  is  a  number  that  expresses  the 
effect  of  a  given  part  upon  system  reliability.  Thus  its  critic¬ 
ality  is  affected  by  both  its  inherent  reliability  and  whether  it 
is  used  in  a  manner  that,  if  it  fails,  the  system  fails.  All 
parts  may  be  ranked  in  order  of  decreasing  criticality,  which 
amounts  to  a  ranking  of  importance  to  system  reliability  achieve¬ 
ment. 

"Levels  of  Essentiality"  have  been  established  by  BuShips  Inatruc 
tion  4410.17  for  ships  piping.  Essentiality  levels  1  and  II  «r« 
roughly  equivalent  to  "critical"  parts  above.  When  the  relia¬ 
bility  and  "essentiality"  of  ships  piping  are  considered  together 
we  are  considering  "criticality"  cf  the  piping. 

The  Bureau  engineer  should  make  sure  that  critical  parts  lists 
and  criticality  rankings  are  used  when  appropriate. 

5.  PARTS  SPECIFICATION 

Many  of  the  contractor's  design  engineers  may  require  the  aama 
part,  and  quite  commonly  its  important  characteristics  are  diff¬ 
erent  for  different  applications.  Thus  if  each  design  engineer 
writes  his  own  specification,  the  parts  manufacturer  is  confused 
by  many  inconsistent  specs.  The  supplier  is  forced  to  form  his 
own  opinion  of  the  relative  importance  of  characteristics  to 
standardize  on  only  one  or  a  few,  to  stay  competitive. 

But  the  contractors  parts  engineering  specilists  are  in  a  far 
better  position  to  evaluate  relative  importance  of  character¬ 
istics,  or  to  talk  the  design  engineer  out  of  a  specified 
characteristic  that,  on  second  thought,  is  not  worth  ita  coat. 

Or  perhaps  the  specialists  can  suggest  an  alternative,  for  tAtich 
the  specification  is  already  available. 

Problems  like  these  have  led  nearly  all  major  contractors  to 
establish  centralized  specification  groups,  with  parts  spaci- 
fication  in  the  Parts  Engineering,  group. 
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Many  parts  suppliers  have  embraced  the  word  "reliability"  without 
knowing  what  it  means.  Quite  a  few  make  "Hi-Rel"  parts  which 
may  have  a  tighter  quality  control  specification,  but  which 
actually  promise  nothing  about  ^he  reliability  or  failure  rate 
of  the  part.  It's  analogous  to  the  small  table  model  "Hi-Fi" 
radios  which  are  not.  The  Bureau  engineer  must  make  very  sure 
that  parts  reliability  (failure  rate,  not  quality)  is  specified. 

A  centralized  contractor  group  responsible  for  writing  all  parts 
specifications  can  (a)  contribute  to  standardization  and  its  in¬ 
herent  reliability  advantage,  (b)  write  specs  around  at  least  two 
suppliers  products  to  preserve  competition  (price  and  reliability), 
(c)  use  the  widest  possible  knowledge  of  parts  deficiencies  across 
projects  to  influence  spec  revisions  and  new  specs,  (d)  make  sure 
that  all  necessary  quality  control  requirements  are  included,  and 
most  importantly  (e)  see  that  maximum  permissible  failure  rate, 
and  the  means  of  verification  (including  confidence  level)  is 
specified,  but  only  to  the  level  actually  needed. 

6.  ENGINEERING  STOCKROOM 

A  commonly-overlooked  indirect  source  of  unreliability  is  the 
engineering  stockroom  from  which  the  design  engineer  or  techni¬ 
cian  selects  parts  that  "will  work"  in  his  mockup,  breadboard, 
engineering  model,  or  engineering  prototype.  At  this  design 
phase  he  is  primarily  interested  in  juggling  parts  and  values 
until  the  mechanism  or  circuit  functions  in  the  manner  desired. 

He  has  not  yet  tackled  many  constraints  such  as  environment , 
tolerances,  reliability,  maintainability,  parts  delivery,  cost, 
etc. 

But  once  these  parts  are  operational  in  the  engineering  model, 
most  of  them  have  a  way  of  being  specified  in  the  parts  list  for 
release,  and  vociferously  defended.  To  challenge  them  seems  a 
reflection  on  the  design  engineer's  judgment,  and  besides  he  is 
now  too  close  to  a  scheduled  completion  to  do  much  chanqinq. 

Many  contractors  avoid  this  problem  by  (a)  having  the  parts 
engineering  specialists  control  the  stockroom  inventory,  (b) 
prohibiting  non-preferred  parts  in  the  stockroom  whenever  an 
equivalent  preferred  part  is  available,  (c)  prohibiting  the 
ordering  of  a  non-preferred  part  in  the  same  situation,  and  (d) 
having  the  parts  engineering  specialists  review  each  design 
engineering  request  for  new  experimental  parts,  and  order  the 
“best"  part  (reliability  and  otherwise)  with  the  design  engineers 
concurrence . 
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7  •  PAKTS  HANDLING 

The  contractors  manufacturing  activity  (and  others  to  a  lesser 
extent)  handle  millions  of  parts,  and  must  do  so  in  an  economical 
and  practical  manner,  bins  are  widely  used  for  "nuts  and 

bolts"  hardware.  It  was  natural  to  use  bins  for  electronic  parts, 
While  most  parts  can  take  the  banning  around  they  get  from  being 
poured  into  a  bin,  or  dropped  on  the  floor  and  tossed  back,  many 
cannot  without  degradation  of  reliability.  For  this  reason,  the 
Minuteman  "high-reliability"  parts  program  utilized  rigid  handl¬ 
ing  procedures. 

There  are  many  other  opportunities  for  mishandling  that  cause 
reliability  reduction.  Examples  are  inadequate  supplier  shipping 
containers,  rough  treatment  in  receiving  inspection,  stacking 
large  quantities  with  damage  at  the  bottom,  fatigue  of  parts 
leads  by  repeated  bending  during  assembly,  overheating  of  sensi¬ 
tive  parts  during  soldering,  overstress  in  factory  tests,  resting 
assemblies  (such  as  circuit  boards)  on  their  parts  while  in 
factory  transit,  dropping  assemblies  on  their  parts,  straining 
part  leads  or  mountings  by  misalignment,  etc. 

Review  of  the  above  shows  that  some  can  be  detected  by  quality 
inspection,  but  many  cannot.  The  latter  cause  eventual  failure 
due  to  operational  vibration,  temperature  cycling,  etc.  There¬ 
fore  much  attention  has  been  given  to  control  of  handling  pro¬ 
cedures  to  prevent  reliability  degradation.  Since  the  parts 
engineering  specialist  is  most  knowledgeable  about  the  sensitive 
characteristics  of  each  part,  most  contractors  have  made  such 
specialists  responsible  for  (a)  the  generation  of  control  policy 
and  procedures,  for  (b)  review  of  engineering  drawings  and 
specifications  to  make  sure  they  are  invoked,  and  (c)  review  of 
factory  and  audit  procedures  to  make  sure  they  are  invoked. 

The  Bureau  engineer  should  make  very  sure  that  parts  handling 
procedures  are  adequate  and  enforced. 

8  •  TRACEABILITY 

When  a  failure  occurs  in  engineering  or  factory  test  or  actual 
operation  of  a  system,  and  failure  diagnosis  (Chapter  16)  shows 
that  the  part  and  not  its  application  is  deficient,  three  steps 
become  mandatory.  They  are  (a)  replace  or  repair  the  deficient 
part,  (b)  take  steps  to  prevent  receipt  or  use  of  any  more  such 
deficient  parts,  and  (c)  replace  or  repair  any  other  such  parts, 
now  questionable,  that  may  have  been  used  anywhere  in  a  manner 
that  system  failure  could  result.  It  is  this  third  step  that  is 


complex  and  of  concern  here.  For  example,  the  use  of  welded  in¬ 
stead  of  seamless  drawn  tubing  on  the  Nautilus  had  to  be  traced. 

The  problem  is  to  keep  records  in  such  a  manner  that  critical 
parts  (whose  failure  can  cause  system  failure)  can  be  traced 
back  to  an  individual  or  lot  test  history,  and  conversely  that 
all  application  of  a  questionable  such  part  can  be  traced  to  the 
specific  systems  and  components  where  it  was  used.  If  not 
arranged  carefully  this  can  result  in  substantial  paperwork  and 
cost  beyond  real  justification.  BuShips  Instruction  4410.17 
discusses  such  control.  There  are  two  basic  techniques,  both 
widely  used: 

9 . 1  LOT  CONTROL 

In  writing  its  part  specification,  the  parts  engineering  special¬ 
ist  requires  the  supplier  to  keep  complete  records  of  the  tests 
(100%  or  sa»ple)  he  conducts  on  each  manufacturing  lot  during 
irfhich  absolutely  no  material  or  process  change  was  made.  If  a 
dhange  was  made  he  must  record  it  and  start  a  new  lot  number. 

Then  every  part  shipped  to  the  contractor  must  be  identified 
(either  on  tne  part  or  by  accompanying  paperwork)  by  its  lot 
number,  or  the  lot  number  of  the  batch  or  lot  of  parts.  Then 
it  Is  up  to  the  contractor  to  handle  them  in  such  a  way  that  the 
record  for  each  serial- numbered  assembly  shows  the  lot  numbers 
of  its  critical  parts.  Complete  system  records  show  the  serial 
numbers  of  all  assemblies. 

1.2  SERIALIZATION 

When  parts  undergo  a  special  high-reliability  manufacturing 
process,  and  are  individually  { 100%)  tested,  and  then  used  in 
very  critical  applications  (part  failure  begets  cert,  in  system 
failure),  they  are  often  given  individual  part  serial  numbers. 
This  of  course  provides  excellent  traceability,  but  requires 
a  great  deal  of  paperwork  and  high  cost.  It  may  be  quite 
practical  and  effective  for  few-of-a-kind  systems,  but  unjusti¬ 
fiable  for  substantial  production,  where  lot  control  is  a  reason¬ 
able  compromise 

Another  problem  with  part  serialization  is  that  many  electronic 
parte  ere  getting  too  small  to  hold  a  number,  so  the  numbers  are 
printed  alongside  them  on  a  card  used  by  the  supplier  to  protect 
and  ship  them.  Dieassociation  between  card  and  part  inevitably 

occurs . 


Serialization  is  of  course  very  widely  practiced  at  the  component, 
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subsystem  and  system  level,  for  roughly  the  same  purpose,  and  is 
very  effective. 

8 . 3  CONTROL 

Since  lot  or  serial  number  instruction  must  appear  on  the  part 
specification  to  the  supplier,  as  well  as  on  release  oaperwoik 
to  manufacturing,  it  is  usually  a  parts  engineering  responsibility 
to  develop  practical  traceability  pdicy  and  procedures.  The  Bur¬ 
eau  engineer  must,  make  sure  they  are  adequate  and  enforced. 

9 .  PARTS  TESTING 

Chapter  11  develops  the  techniques  for  verifying  reliability. 

The  standard  MTB4F  test  ("sequential  life  test"  in  the  language  of 
statistics)  can  work  very  well  for-  most  components,  subsystem* 
and  systems,  depending  upon  the  importance  of  reliability  and  the 
economics  of  proving  it.  For  some  parts  it  works  well,  and  for 
others  it  is  impractical. 

Today's  part  failure  rates  run  the  order  of  0.002  to  1.0  failuras 
per  million  hours  of  stress,  which  is  1  million  to  500  million 
hours  KTBF.  If  1000  of  each  part  is  life- tested  to  lO-times-ffTBF , 
the  test  durations  will  rur.  1.1  to  57  0  years .  Obviously  the 
higher  the  part  reliability  the  more  impossible  it  is  to  test  for 
it.  They  become  obsolete  long  before  they  can  be  verified.  Aftd 
the  test  investment  is  not  justifiable. 

Standardization  on  certain  preferred  parts  by  the  contractor,  and 
particularly  if  many  contractors  can  agree  on  such  preferred  parts, 
permits  a  good  supplier  to  greatly  increase  his  volume  of  each  of 
fewer  parts.  Thus  he  can  justify  automatic  equipment  and  lifa 
testing  of  parts  that  otherwise  would  remain  unveri  fied. 

A  very  significant.  contriDution  to  the  parts  testing  proble*  is 
the  Naval  Ordnance  Laboratory  (NOL)  Inter-Service  Data  Exchange 
Program  (Il>EP)  which  collects  well-organized  test  data  from  «arry 
participating  contractors,  digests  it,  and  issues  periodic 
reports  on  file  cards  with  microfilm  inserts  of  tha  detailed 
reports.  Contractors  frequently  find  an  IDEP  test  report  suffi- 
cent  to  eliminate  costly  tests  that  would  otherwise  be  required. 
The  system  works. 

Knowledge  of  the  above  verification  situation  surrounding  each 
part  uaed  in  industry  is  generally  highest  in  the  contractor’s 
Parts  Engineering  group.  Therefore  most  contractors  assign  to 
then  the  responsibility  to  (a)  determine  what  parts  ver i fi- 
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cation  approach  is  best,  (b)  specifying  tne  exact  test  procedure, 
(c)  determine  whether  the  supplier  can  best  do  it,  (d)  actually 
conduct  the  tests  if  the  supplier  should  not,  (e)  interpret  the 
test  results  for  acceptance  or  rejection,  and  (f)  work  with  the 
supplier  on  deficient  parts.  The  Bureau  engineer  should  make 
sure  that  such  integration  of  reliability  test  activities  is 
adequate . 


10. 


SUMMARY 


Although  perfect  parts  do  not  assure  perfect  system  reliability, 
the  failure  of  any  one  of  most  parts  of  a  system  causes  system 
failure.  Therefore  thorough  engineering  control  of  r  arts  is 
essential  to  system  reliability. 

Most  contractors  have  found  a  centralized  Parts  Engineering 
activity,  with  experts  in  specific  kinds  of  parts,  to  be  in¬ 
dispensable.  This  chapter  has  outlined  the  principal  activities 
of  such  specialists,  why  they  are  needed,  and  what  they  con- 
tr ibute . 


The  basic  contribution  is  establishment  of  preferred  parts  for 
use  by  all  design  engineers,  and  ample  consultation  and  guide¬ 
lines  for  part  selection  and  application. 

Parts  data,  particularly  concerning  reliability,  is  needed  in 
concise  form  for  design  engineer  decisions.  Fart  specification, 
handled  uniformly,  pays  off  in  reliability.  Engineering  stock- 
room  control,  pa: ts  handling  procedures,  traceability,  and  parts 
testing  have  become  especially  important  as  higher  system  relia¬ 
bility  is  required. 

No  amount  of  system  design,  analysis,  and  reliability  production 
will  bring  high  reliability'  unless  there  is  controlled  and 
adequate  critical  parts  reliability.  The  Bureau  engineer  must 
assure  himself  that  the  contractors  procedures  and  enforcement 
thereof  are  adequate  for  the  reliability  required. 
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Chapter  19 

SUPPLIER  RELATIONSHIPS 


In  order  to  design  and  manufacture  equipment  and  systems  of 
known  and  controlled  reliability  and  maintainability,  the 
contractor  must  in  turn  knew  and  control  the  reliability  and 
maintainability  of  what  he  buys.  His  sources  may  be  called 
vendors,  distributors,  suppliers,  subcontractors,  etc.,  but  we 
use  the  broad  word  "suppliers"  to  mean  all  of  then. 

Since  suppliers  normally  provide  the  bulk  of  deliverable  hard¬ 
ware,  it  is  imperative  that  BuShips  reliability  and  maintain¬ 
ability  requirements  be  passed  on  to  suppliers  appropriately. 

But  that  weasel  word  is  a  Pandora's  box,  because  the  depth  of 
specification  and  control  must  vary  widely  with  (a)  the  relia¬ 
bility  and  maintainability  actually  needed,  (b)  the  gap,  if  any, 
between  needs  and  state  of  the  art,  and  (c)  the  design  level 
(parts  vs  systems).  Obviously  a  competent  pipe  manufacturer 
would  not  and  should  not  hold  still  for  a  complex  reliability 
and  maintainability  program  plan  of  15  tasks.  Conversely  a 
radar  computer  manufacturer  had  better  expect  to  guarantee  the 
MTBF  and/or  agree  to  a  set  of  reliability  and  maintainability 
tasks . 

While  supplier  quality  control  programs  have  been  very  well 
developed  by  many  contractors,  and  some  of  these  claim  to  control 
reliability,  the  fact  is  that  very  few  actually  get  into  the 
suppliers'  design  capability  to  achieve  specified  reliability 
and  maintainability.  And  there  is  no  other  place  to  get  it. 

Tight  surveillance  of  a  supplier's  procedures  and  quality  control 
can  minimize  reliability  and  maintainability  degradation,  but 
cannot  assure  design  achievement  thereof. 

Chapter  22,  Section  15,  and  Chapter  23,  Sections  2  and  3 
delineate  the  BuShips  steps  necessary  to  assure  that  the 
contractor  establishes  an  adequate  supplier  control  program. 

1.  SUPPLIER  QUALIFICATION 


Without  going  to  the  expense  of  a  personal  survey,  contractors 
can  find  out  quite  a  lot  through  the  mail.  So  can  BuShips,  for 
that  matter.  Figures  19-3  and  19-4  show  a  2-page  questionnaire 
that  will  pretty  well  highlight  what  suppliers  have  done  and  are 
doing.  Another  page  could  be  added  suggesting  that  the  supplier 
state  what  he  is  planning  to  do,  but  of  course  such  words  are 
cheap. 
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Mail  the  completed  questionaire  to 
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Such  returned  questionaires  permit  the  contractor  or  BuShips  to 
(a)  eliminate  obviously  non-competitive  suppliers,  (b)  select 
those  to  be  considered  (of  which  ultimately  the  serious  conten¬ 
ders  will  be  personally  surveyed) ,  and  (c)  determine  about  how 
much  reliability  and  maintainability  education  and  control  he 
will  have  to  do  to  get  the  required  reliability  and  maintainabil¬ 
ity  in  supplier  products. 


2.  SUPPLIER  REQUIREMENT  CRITERIA 


Just  as  the  contractors  reliability  and  maintainability  program 
is  determined  by  the  BuShips  requirements  as  in  chapter  23 
sections  1,  2.1.1,  3.1,  so  must  the  supplier's  reliability  and 
maintainability  program  be  determined  by  the  contractors  system 
or  equipment  reliability  and  maintainability  requirements.  The 
contractor  had  to  apportion  them  3s  in  chapter  6,  and  has  deter¬ 
mined  what  quantitative  reliability  and  maintainability  he  must 
specify  to  suppliers.  Chapter  22  section  15  details  the 
BuShips  instruction  to  contractors. 

But  not  all  specifications  to  suppliers  need  or  can  have  such  a 
quantitative  requirement.  Here  are  the  factors  to  consider: 

2 . 1  NEED 


If  the  contractor  is  supplying  to  BuShips  equipment  whose  failure 
would  not  have  important  consequences  (maybe  air  conditioning 
equipment),  BuShips  would  not  specify  quantitative  reliability 
nor  a  reliability  and  maintainability  program.  So  obviously 
neither  would  the  contractor  to  his  suppliers.  Conversely  fail¬ 
ure  of  a  system  upon  which  a  major  operational  task  may  depend 
(say  the  power  supply  for  fire  control)  would  require  the  con¬ 
tractor  to  apportion  BuShips  quantitative  requirements  to  his 
suppliers . 

2 . 2  TECHNOLOGY  GAP 

If  the  well-verified  need  is  for  a  system  of  much  higher  relia¬ 
bility  (say  100-to-l  MTBF  improvement)  than  previously  achieved, 
and  this  is  not  exactly  uncommon,  then  the  contractor  can  only 
get  it  by  relatively  heroic  measures  as  outlined  in  chapter  13. 
But  quitr  commonly  it  is  a  judicious  choice  between  (a)  redun¬ 
dancy  of  mode  or  components  and  (b)  parts  improvement,  the  latter 
being  very  expensive  if  not  infeasible.  But  the  redundancy  can 
be  within  his  own  design  us’ng  supplier  components,  or  within 
the  supplier  components,  or  both.  In  any  case  he  will  most 


certainly  have  to  specify  quantitative  reliability  to  his  sup¬ 
pliers,  to  a  much  lower  design  level  than  would  otherwise  be 
justified. 

3.3  CRITICALITY 

If  the  contractors  entire  system  has  a  certain  anticipated  fail¬ 
ure  rate  (say  100  failures  per  million  hours  of  operation) ,  then 
a  simple  measure  of  the  “criticality”  of  a  component  thereof  is 
the  increment  of  that  system  failure  rate  (say  2  failures  per 
million  hours)  due  to  that  component.  If  the  component  is  used 
redundantly,  its  own  failure  rate  would  be  higher  (say  10  fail¬ 
ures  per  million  hours). 

Now  obviously  a  component  of  zero  criticality  (say  a  pilot  light 
whose  failure  would  not  contribute  to  system  failure  rate)  does 
not  need  a  quantitative  reliability  specification,  much  less  a 
reliability  program.  On  the  other  hand  components  of  high  cri¬ 
ticality  (say  a  radar  magnetron)  demand  quantitative  reliability 
specification  to  suppliers  if  the  contractor  is  to  achieve  and 
control  his  system  reliability. 

So  the  contractor  must  rank  all  components  in  the  order  of  de¬ 
creasing  criticality,  and  there  will  be  a  place  in  the  list  below 
which  quantitative  reliability  specification  and/or  reliability 
and  maintainability  programs  cannot  justify  their  cost  and  pos¬ 
sible  delivery  delays.  One  way  to  locate  this  boundary  is  by 
trial  and  error,  starting  at  a  low  value  (in  the  above  example, 
ask  for  supplier  quotations  based  upon  quantitative  specification, 
verification,  and/or  control  for  all  criticalities  above  1  failure 
per  million  hours).  Then  as  the  supplier  cost  estimates  thereof 
come  in,  adjust  the  boundary  upward  to  an  economically  justified 
level  (say  5  failures/million  hours). 

2.4  SUPPLIER  EXPERIENCE 

Quite  commonly  an  excellent  supplier  will  have  had  little  or  no 
experience  with  design  for  specified  reliability.  If  he  happens 
to  be  a  sole  source  for  the  needed  component,  or  all  other  sources 
are  not  interested  in  upgrading  their  capability,  the  contractor 
has  no  choice  but  to  bring  the  selected  supplier  up  to  speed.  In 
this  case  he  will  undoubtedly  specify  specific  tasks  to  be  done, 
to  achieve  a  specified  reliability  and  maintainability. 
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3. 


SPECIFICATIONS 


It  is  very  obvious  that  the  supplier  cannot  be  held  contractually 
responsible  for  what  the  contractor  failed  to  specify  explicitly. 
And  yet  it  is  impossible  to  specifv  everything.  The  contractor 
has  to  select  experienced  and  trustworthy  suppliers  and  bank  on 
their  own  desire  to  satisfy  him  in  order  to  stay  in  business. 

But  there  are  grey  areas.  If  the  contractor  plans  to  use  a 
suppliers  product  in  a  new  way ,  the  supplier  cannot  be  expected 
to  forsee  the  implications  without  detailed  engineering  attention 
to  the  application.  This  is  particularly  true  of  reliability  and 
maintainability  implications . 

3.1  SUPPLIER  SPI  JIFICATIONS 

Most  suppliers  develop  excellen  specifications  on  their  products. 
After  studying  such  specs,  and  judging  how  well  the  supplier 
^tays  within  his  specs,  customers  then  place  orders  by  product 
(or  catalog,  model  or  part)  number  appearing  on  the  spec.  But 
the  customer  in  so  doing  is  taking  a  risk  that  (a)  the  product 
that  he  . e  eives  will  not  be  within  specs,  (b)  something  not 
quantified  in  the  spec,  that  the  customer  assumed  would  be 
acceptable,  does  not  fit,  and  (c)  the  supplier  has  changed  some¬ 
thing  that  dc  s  net  affect  the  spec,  but  does  affect  the  customers 
applic  .ion  adversely.  It  is  the  last  two  points  that  often 
adversely  affect  reliability. 

As  a  corv-equence,  most  contractors  have  developed  internal  rules 
that  say  concerning  components  of  criticality  above  the  boundary 
in  2.3  above,  that  (a)  no  critical  component  may  be  ordered  by 
supplier  product  number  alone,  (b)  the  suppliers  specification 
may  be  referenced  if  uniquely  numbered  and  dated  for  each  change 
thereof,  or  it  may  be  copied,  and  (c)  the  contractors  additional 
spec  requirements,  if  any,  must  be  stated,  as  outlined  below. 

3.2  CONTRACTOR  SPECIFICATIONS 


For  best  economy,  delivery,  and  reliability  of  supplier  products, 
it  is  always  desirable  to  order  what  the  supplier  considers 
standard  ("off-the-shelf"),  which  implies  using  only  the  suppliers 
specification.  But  this  may  or  may  not  provide  adequate  relia¬ 
bility.  Even  if  it  does,  the  supplier  may  make  a  design  change 
that  he  considers  a  r^  al  product  improvement,  but  because  of  the 
way  it  is  used  by  the  contractor,  reliability  degration  may 
result . 

Therefore  contractors  may  adopt  internal  rules  that  say  in  addi- 
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tion  to  those  in  3.1  above,  (a)  list  the  actual  application  spec 
requirements,  including  reliability  and  maintainability,  but 
excluding  unnecessary  feature  or  spec  ranges  that  available 
products  may  possess,  (b)  use  the  supplier  spec  ranges  and  words 
wherever  they  fit  the  need  very  well,  and  identify  them  as 
supplier  standard,  a..'1  (c)  call  the  suppliers  attention  to  what, 
in  his  standard  spec,  not  needed.  This  will  permit  the 
supplier  to  consider  tradeoffs  with  the  things  that  are  needed 
beyond  his  standard  spec,  resulting  in  minimum  cost  and  delay. 

3 . 3  DATA 


Good  suppliers  are  ready  to  do  anything  that  is  economically 
justified  to  improve  their  products  in  the  eyes  of  their  custom¬ 
ers.  Thus  it  is  not  difficult  to  convince  most  of  them  that 
there  should  be  a  two-way  flow  of  data  on  their  products.  This 
does  not  refer  to  QC  corrective  action  procedures  when  a  suppli¬ 
ers  product  gives  trouble  in  the  contractors  plant.  It  does  ref'1' 
to  exchange  of  reliability  and  maintainability  data. 

The  supplier  should  be  requested  tc  provide  full  reliability  and 
maintainability  data  on  the  produces  nat  he  is  furnishing  during 
the  suppliers  design  and  manufact’  ~e  thereof,  including  data 
derived  from  other  customer  usage  o  such  products.  The  data 
should  include  operating  (stress)  time  or  cycles,  local  environ¬ 
ment  and  stress,  failure  rate  (or  %  fa  led,  if  failure  ’■ate  is 
not  obtainable),  and  downtime  and  manhours  per  failure.  Other 
data  may  also  be  appropriate  for  specific  products. 

In  return  for  the  above  (perhaps  at  no  extra  cost)  the  contractor 
should  agree  to  provide  the  same  information  to  the  supplier  on 
his  products,  throughout  contractors  design,  manufacture,  tt"' , 
checkout,  and  operation.  And  if  BuShips  will  provide  such  it 
formation  to  its  prime  contractor  after  delivery  and  contract 
completion,  the  contractor  should  agree  to  continue  the  inform a 
tion  flow  to  suppliers.  This  is  probably  the  most  powerful  means 
of  assuring  that  successive  supplier  designs  will  have  the  re¬ 
quired  reliability  and  maintainability. 

3.4  REQUEST  FOR  PROPOSAL 

As  contractors  find  it  necessary  to  impose  quantitative  reliabil¬ 
ity  and-maintainability  requirements  on  supplier-,  and  particu¬ 
larly  as  we  enter  the  era  of  attention  to  cost-effective  design, 
the  simple  speci fication  to  suppliers  will  no  longer  serve  the 
purpose  for  many  products.  The  contractor  will  not  know  what 
reliability  and  maintainability  can  be  achieved  until  he  tells  a 
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few  suppliers  what  he  wants,  and  asks  for  their  well-considered 
formal  proposals.  Nor  will  he  know  what  the  desired  reliability 
and  maintainability  may  cost.  But  with  such  proposals  he  can 
trade  off  the  achievable  reliability  and  maintainability  with 
cost,  including  consequent  Ownership  cost  (see  chapter  26),  and 
decide  what  exactly  to  require  of  the  supplier. 

The  content  of  a  BuShips  Request  for  Proposal  (RFP)  is  covered 
in  chapter  23  sections  2.1  and  3f  and  the  principles  are  no 
different  for  a  contractors  RFP  to  suppliers.  If  the  contractor 
for  a  fire-control  system  is  subcontracting  its  radar/computer , 
he  will  probably  use  nearly  all  the  elements  of  section  2.1  of 
chapter  23  including  a  comprehensive  reliability  and  maintain¬ 
ability  Program  Plan.  But  if  he  is  buying  roller  bearings,  he 
may  use  a  very  simple  RFP  involving  no  Program  Plan  specifica¬ 
tion  of  "how"  the  supplier  will  achieve  the  desired  reliability. 


4.  PROPOSAL  EVALUATION 

Again  BuShips  proposal  evaluation  is  covered  in  chapter  23 
section  4.,  and  the  principles  are  the  same  for  contractor  eval¬ 
uation  of  supplier  proposals.  The  contractor  can  use  the  same 
weight  ratings  that  were  used  to  evaluate  his  own  RFP,  or  a 
different  weight  rating,  according  to  the  contractors  judgment 
of  relative  contribution  to  reliability  ^nd  maintainability. 

5.  SURVEYS 

Mos  contractors  have  appropriate  supplier  survey  procedures, 
usually  conducted  by  the  Quality  Control  group.  However,  most 
of  these  do  not  actually  evaluate  the  suppliers  capability  to 
dcs ; gn  for  specified  reliability  and  maintainability,  but  rather 
his  " control"  procedures. 

The  Supplier  Questionaire  in  1.  above  of  course  constitutes  a 
preliminary  survey,  but  it  cannot  generate  enough  confidence 
upon  which  to  base  a  decision  to  place  an  order. 

t.hen  the  contractor  has  narrowed  his  decision  down  to  a  choice 
1  atween  say  two  suppliers  of  critical  components,  it  is  time  to 
conduct  pvrson-to-person  surveys  of  the  two  to  aid  in  the  final 
decisions.  Chapter  24  section  6.1  discusses  such  BuShips  sur - 
"eys ,  and  the  same  procedures  can  be  used  by  contractors. 

A^ter  placing  an  order  with  a  supplier,  the  contractor  should 
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then  conduct  periodic  surveys  of  his  suppliers  of  critical  com¬ 
ponents.  The  selection  of  suppliers  to  be  surveyed  and  the  fre¬ 
quency  of  surveys  (usually  every  6  to  12  months,  or  when  the 
supplier  has  made  a  significant  'hange  of  design,  manufacturing 
method,  or  organization)  depends  upon  the  criticality  of  the 
component.  Chapter  24  section  6.4  outlines  the  BuShips  post¬ 
award  surveys,  and  contractor  procedures  would  be  the  same. 

The  next  two  pages  show  a  Design  Vendor  Appraisal  Summary  form 
used  by  a  major  shipbuilder  (1),  following  which  are  the  detailed 
questions,  pertinent  to  reliability  and  maintainability,  used  by 
the  appraiser. 


6.  CONTRACT  NEGOTIATION 

Considering  the  necessity  for  the  contractor  to  require  specified 
reliability  and  maintainability  from  his  suppliers  of  critical 
components,  and  the  fact  that  he  will  want  to  pass  along  some  of 
his  BuShips  incentive  to  his  suppliers,  there  is  likely  to  be 
more  negotiation  of  supplier  contracts  than  in  the  past.  Basic¬ 
ally  he  should  be  negotiating  promised  reliability  and  maintain¬ 
ability  values  vs.  their  cost,  without  trying  to  control  the  wav 
in  which  the  supplier  gets  there.  But  with  the  many  excellent 
suppliers  who  have  not  yet  had  experience  with  the  reliability 
and  maintainability  tastes  needed  to  get  there,  the  contractor 
must  also  negotiate  tasks. 

The  BuShips  contract  negotiation  principles  are  covered  in 
chapter  23  section  2.4,  and  chapter  24  section  4.  These  can  be 
appropriately  interpreted  tor  contractor  use  with  suppliers. 


7  *  SUPPLIER  EVALUATION 

After  a  contract  is  placed  with  a  supplier,  the  contractor  will 
need  to  check  periodically  whether  the  supplier  is  still  on  the 
track.  F^r  a  substantial  subcontract  of  a  critical  component 
this  may  involve  monthly  (a) review  of  the  suppliers  monthly  pre¬ 
diction  of  reliability  and  maintainability  based  or  design  to 
date,  (b)  review  of  the  quantitative  requirements  if  prediction? 
are  not  consistent  therewith,  (c)  review  of  supplier  progress  on 
reliability  and  maintainability  Program  Plan  tasks,  particularly 
■Corrective  Action  Log  progress,  (d)  review  of  verification  test 
results,  and  (e)  a  re-survey  every  6  to  12  months  of  the  sup¬ 
pliers  capability. 
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COOPERATION  DC:  NECESSARY  DESIGN  CHANGES 


THIS  COLUMN  IS  FOR 
PURCHASING  USI  ONLY 


NO  EXPERIENCE 


CHAN««  |«AIUAflON|  AM  HirQIUIO  >INOOC  tHOwt  IILUCTanCI  to 
CUE*RNflV  WM|N  MOOlTlCAflOfrt  TO  CHAN*|  (**l*Nt  H|  M»  ll||  MOO 
D4U4MS  a 41  tlOUiMO.  TH*1  INCLUM1  0*n*  MFORI  mOOTiCaI'CM  (vaLUa 
MAlIRlAl  lUKfiON  AMO  COW*AtARU  IiOnS  Alt  IMCuMD  COOP!  ■  ATEON 

ll»  VIMOOI  OFFItS  LU**IITl<</«S  M  It  JNOWN.  »UT  It  SLOW  IN  COM*** 

MAlllU.ir  AND  MaCiICAIUiTT  ©f  USUALLY  OfMANOt  IHR  A  COMMNtA 

SUCH  MOO*F*CATl©Nl  YEN  OCA  it  NOT  HON 
OPPORTUNIST  AlOilT  MMANOIMO 
»*  Cl  INCMAMt  ON  CNAN4I 

nuiu. 


MODIFICATION  IVALUATlOT  S  ARI  POOR  j 
TlNDO*  HAL  ROITNTlAi  tul  NOT 

OISill  TO  UMOrRTaRI  ML4N 
MOOIFlCAttONS  unit  ON  MOST 
CH«N*lt  At  OPPORTUNITY  TO  MAM 
monit 

TINOOR  WHOLLY  UNCOOP%RaTIV|  If 
MU»M  UOO«»ICATK>N J  IlfHlR  NOT 
CAPaKR  OR  UMWIlilF**  TO  FIR  FORM 
to  standards  RIQUiaiO.  UT'.IZIS 
ORSJ*N  CMAN4I  AT  A  TOOL  FOR 
UORRI'ANT  NRCMaMO  FRKI 

ORMANCS. 

1  3  1 

i 4 1 

H.  At  HERENCE  TO  r'JACHAS*  REQUIREMENTS,  PLANS  i  SPECIFICATIONS 


NO  EXPERIENCE 


*WUH  TO  •UtCHAtl  0*0(1 
It  JU>R|U|Nlt  ANO  LMCHlCAUONt 
t«ALUAt|t  ANO  COMPARtf  TMlM  WITH 
tiUiLAt  COMP©*!*!  NltTOlIlt 


IWtt  U*  TO  AtniCAlU  RTOUlrtMINTl 
WITH  OCCASIONAL  MINOR  MVtATlON}, 
TINOOR  It  COOMMTivt  II  THI 
•  ItOLUTtON  Of  SUCH  MfOtMCilt 


O^i.-ONt  ARIAS  O#  UNCIMAMTT  ANO  nCCAWOPFRUT  TAILS  TO  TlANSMST 


anOLTIS  THlM  m  SAtSFACtORT  WCISSaM  'NjtluCHONS  MOM  wm 

MaHNIR  OUSlMlNATlON  Of  FUICMASI  t©  AM  iMfiOTH  «NO  HMDS  IH|M 
OfOia  OOCUMfNlt  WITHIN  TN| 

TlNDOt »  rlani  If  -DIQoaTi 


TIG  LATH  tRtClftCAT’OM  aiOUitlulNTS  I  *l»iOoa  UNtAtUSACTOa*  NO  IMRaOTl 
UNIL  Alia  ALLY  UNCO  OH«AllV|  •«  >  TN»  [  M|N1  NOTED  AM|B  VENDOR  WARNIO 


aitcuDHOH  of  oitcaifANCut  holds 
f  O  DOCUMINTS  IN  OffICI  If AVWSO 
tHOR  OR  OTHlIt  UNiNfrNVIO  Of 
Of>A«i  RtouiaiMttm 


aROuT  POOR  COONRAlKM  Dots 
NOT  I  TIN  Rt  AO  R  o  OOCUMlNIt 
ANO  MARIS  UNWaRRaNTIO  AttUMR 
HONS  ABOUT  WHAT  |  IN  TH|M 


~TT1 


f.  DESIGN  SERVICE 


NO  EXPERIENCE 


TtNOOBS  01  WON  IKFBITf  MVIIW 
IB  DivrWON  RBOtUMt  At  A  SROUF 
TINOOR  MAS  AN  AOIOUATI  |M«<NIIR 
INC.  CHICK***  HOCUS  f ACTORS 
SUCH  At  MAINTINaiVCI  ANO  SRaB| 

Parts  RtctiTl  Careful  stuot 
urv<i  it  (icmiNf 


IM4NIIRS  Rtf  CARaHI  BUI 
lIRlRltNCI  MINOR  CHMICuiftU 
*1  ROBMIN*  (VALUATIONS  ANO 
MAUN*  SOUNO  IN*Mtl|tlN* 
OtCtWONi  D*v do  O*  RaITT  lINi 
INSTRUCTION 


(KftlNIlBS  AM  RILVCTANT  o *  UN. 
AlLl  TO  MAKI  CONWtllNTLT  tOUNO 
(VALUATIONS  ANO  TICHNlCAl 
OtOtiONS  WHICH  am  tURROailO 
IT  THIIR  OWN  MANA*|M|Nt 


IMAINHRIH*  uavtcis  am  UNSATlt. 
RACTOBT.  TINOOR  It  NOT  CAR  ABU 
OR  MMONtTRATIN*  ROUTIvl  ACTION 
MAIMTINAMCI  ANO  SRaR|  RAMS 
RKOMMINOAftOMt  HAVE  1*0 
TECHNICAL  lAtlS 


J.  SALES  ENGINEERING  SERVICE 


mnoor  aitRONSi  to  mcuists  fob 

INFORMATION  It  PROMPT  COMPU  a. 

NO  EXPERIENCE  "OCO"K' 


SIRViCI  it  OCCaWONaut  DiS 
OR*ANl7tO  CAUt***  C  .INFUSION 
ANO  MLATt 


Mw<»  if  UNSATISFACTORY 

iniopmaTiOn  FUBNISHIO  It 

RIHATIOIT  INCOMRUTI  tNCORBICT 
ANO  OR  UNRIASONAUT  MLAY|D 


K.  CO-OPERATION  RE:  <  CIFtC  NONDESTRUCTIVE  TEST  REQUIREMENTS 


NO  EXPERIENC 


INCIFNOUt  A  ROUT  SU**4STINC  1  IW 
ANO  MTTIR  Wav |  OR  TftTiP**  FOR 
Oil  All  TT  Af  MINIMAL  IIHNU  TO 
ROTH  FARMS 


TINOOR  IIRIRlIFfCIt  OTfflCULMI 
ATTlMRTlN*  TO  COMRVT  WITH  HOT 
RtOuifllUlHTS  CaUSIS  MINOR 
MLATt 


TINOOR  It  (FMIMRNMCIO  in 
OUALITT  ANO  PROOF  OF  T«tT 
RIOUiRIMMm  RIOJIIIO  RT  IR  c 


TINOOR  is  UNSATISFACTORY  M«  IS 
NOT  WULIN*  OR  CARARil  O*  Ml  Tin* 
IR  OiTItiON  NOT  RKHItaiMHvn 


U  ^-OPERATION  RE:  f.*ECIFIC  SU (MARINE  APPLICATION  REQUIREMENTS:  LOW  NOISE  LEVEL  REQUIREMENTS.  MERCURY  PROHIIITION,  WEIGHT  MONITORING. 
SmOCK  DESIGN 


NO  EXPERIENCE 


TINOOR  UNSATISFACTORY  FAClllTltt 
APR# a!  TO  U  NHMRUATI  aRRuCaHI 
Mlt  SHCS  ANO  OR  ARRROVIO  TltllN* 
PROCE  DORIS  aRI  NOT  aOh|RIO  TO 
Till  II SUITS  ANO  IRCHNICAI  ANAL- 

tvs  ari  rihat:mt  poor 


CONSIDERING  THE  FO'  lllUTY  OF  MAJOR  AWARDS,  PLEASE 

INDICATE  YOUR  aITiTUDE  toward  DOING  IUSINESS  WITH  THIS  VfcNOOR 

OVTS'  NOINGVENOOR 

BETTER  THAN 

MOST  VCNOORS 

GENERALLY 

CAPABLE 

CAPABLE  BUT 

NEEDS  PROOOING 

HAS  SHORTCOMINGS 
BUT  IMPROVEMENTS 

UNSATISFACTORY 

1 _ a 

□ 

□ 

□ 
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APPRAISERS  QUESTIONS  FOR  THE  DESIGN  VENDOR  APPRAISAL  SUMMARY 

A .  ’echnical  Adequacy  of  Designs 

What  test  and/or  performance  histories  are  gathered? 

J.  How  rigorous  are  calculations  for  performance  and  strength 
disability? 

3.  How  close  to  the  state  of  the  art? 

4.  What  are  the  limiting  feature  of  the  design  for  durability? 

5.  Does  vendor  document  TFR  incidents?  Are  these  available 

to  (the  contractor)?  ; 

B.  Practicability  of  Designs 

1.  Vendor  has  reviewed  his  product's  application  to  sub¬ 

marine  system  and  environments? 

2.  Vendor  has  visited  ships  or  mock-ups  to  confirm  and 

concur  adequacy  of  application  in  his  opinion? 

3.  Is  design  simplicity  exploited  as  a  creativity  character¬ 

istic?  How  is  this  developed? 

4.  How  is  durability  and  reliability  judged? 

5.  How  has  vendor  used  the  stated  design  targets  for  life 

expectancy?  What  technical  proofs  of  compliance  have 
been  made?  ,j. 

6.  Has  vendor  used  maintainability  as  a  design  characteris¬ 

tic?  How? 

7.  What  accessibilities  have  been  provided  in  the  product? 

How  is  the  physical  space  evaluated  from  a  man/main¬ 
tenance  coordination  in  the  ship  environment? 

8.  Are  special  tools  or  alignment  jigs  or  special  sequential 

assembly  checks  required  for  tear  down  and  repair? 

9.  What  are  the  limiting  wear  and/or  deterioration  consider¬ 

ations? 

10.  What  spare  parts  are  proposed  to  suit  continuity  of 

performance  expectations? 

11.  What  repair  schedule  is  designed  to  provide  adequate 

preventative  meaintenance  for  extended  life? 

12.  What  program  of  Design  Review  is  proposed  for  what 

stages  of  design  and  manufacturing? 

’3.  How  will  vendor  invite  (contractor)  participation  in 
Design  Review? 

1 4.  What  means  are  provided  for  proper  material  selection 

and  continuity  of  pedigree? 

15.  What  means  are  provided  for  assurance  of  design,  material, 

and  quality  of  second  tier  purchases? 

C.  Safety  Margins 

1.  What  performance  margin  exists  over  guaranteed  rating? 

2.  What  is  the  "worst  case"  or  limiting  parameter  in 

estimating  type  of  casualty  on  overload? 
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3.  Is  a  failure  analysis  study  made?  What  mode  and 

mechanism  contributes  to  failure? 

4.  Is  there  any  test-to-failure  data  available? 

5.  What  safety  factors  are  standard  practice? 

6.  Are  safety  margins  calculated? 

7.  Can  safety  margin  and  performance  margin  be  related 

to  expected  performance  life?  What  is  the  limiting 
margin  which  degrades  with  operating  time? 

8.  What  Services  (oil,  water,  grease,  electricity,  torque, 

etc.)  variabilities  can  be  tolerated  within  guarantee 
performance  range? 

9.  Is  a  tolerance  analysis  accomplished? 

10.  What  effects  are  expected  if  adverse  tolerance  buildups 

occur?  Is  there  adjustment  means? 

11.  What  overall  performance  factors  or  ranges  of  variabil¬ 

ity  are  controlled  by  permanent- or  maintenance  adjust¬ 
ments? 

12.  What  areas  of  the  design  have  the  maximum  risk  to  suc¬ 

cessful  per forman-e? 

13.  If  vendor  could  afford  to  redesign  and  reschedule, 

what  design  changes  would  he  make  as  improvements? 

D.  Initiative  in  Proposing  Improvements 

1.  Will  vendor  orient  Design  Reviews  toward  improving 

design  as  well  as  justifying  current  work  status? 

2.  Does  vendor  have  the  courage  to  make  the  effort  required 

for  significant  improvement? 

E *  Initiative  in  Cost  Reduction  Through  Value  Analysis 
1.  Can  the  vendors  evaluate  cost-effectiveness  and 
comparison  evaluations  on  their  products? 

F.  Promptness 

1.  Does  vendor  promptly  accept  responsibility  for  obvious 
faults  even  if  evasion  is  available  via  strict  inter¬ 
pretation  of  specs? 

G.  Co-operation,  etc. 

1.  (a)  Are  material  selections  offered  to  (contractor)  for 

agreement? 

(b)  Are  substitutions  made  without  (contractor)  knowledge? 
I.  Design  Service 

1.  Does  vendor  use  planned  objective  Design  Review  at  the 
purchase  order  acceptance  phase? 

2  (a)  How  is  maintainability,  accessibility,  producibility , 

" Inspectability"  determined  as  early  inputs  to  the 
designer? 

(b)  How  is  maintainability  related  to  realistic  pro¬ 
visioning  proposals  at  the  design  stage? 
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K.  Cooperation,  etc.  Non-Destructive 

1.  (a)  How  carefully  does  vendor  extract  all  information 

from  required  tests? 

(b)  Poes  he  attempt  to  vary  performance  parameters  to 
determine  overall  variability  criteria? 

(c)  Does  he  consider  extrapolation  of  performance  with 
time? 

(d)  Does  vendor  design  and  prepare  a  test  plan? 

(e)  Is  (contractor)  party  to  pre-approvals  of  testing 
arrangements  ? 

2.  (a)  Are  test  results  provided  to  Engineering/Design? 

(b)  Are  these  results  checked  agcinst  those  expected  by 
Design  Engineers? 

(c)  Are  differences  evr  Luated  and  incorporated  in 
technical  data  as  lessons  learned  for  future  work? 

For  a  relative  small  purchase  order  for  a  critical  component 
already  designed,  and  whose  reliability  is  already  well-estab¬ 
lished  and  adequate,  the  contractor  would  need  only  (a)  periodic 
confirmation  of  the  predicted  reliability  (which  would  be  subject 
to  data  coming  in  from  all  the  suppliers  customers)  and  (d)  re¬ 
view  of  verification  test  results  prior  to  shipment. 

The  above  review  is  accomplished  by  contractually-arranged 
monthly  reports,  reviewed  in  detail  and  analyzed  by  a  contractor 
reliability  and  maintainability  specialist.  But  such  review 
must  be  supplemented  by  regular  visits  to  the  supplier  for  under¬ 
standing  of  the  problems  and  formal  or  informal  audit  of  the 
suppliers  methods. 


8 .  SUPPLIER  CONTROL 

If  the  supplier  is  competent,  and  has  agreed  to  furnish  specified 
quantitative  reliability  and  maintainability  for  his  product, 
and  his  contract  says  he  will  not  get  paid  if  he  does  not  meet 
the  spec,  no  further  controls  should  be  necessary.  A  competent 
supplier  will  control  himself,  and  the  contractors  actions 
should  always  try  to  get  him  to  do  so.  Then  the  contractor  need 
only  watch  and  regularly  evaluate  the  reports. 

If  the  supplier  of  a  new  and  untried  design  cannot  agree  to  so 
guarantee  reliability  and  maintainability,  then  there  must  be 
contractual  agreement  on  the  tasks  the  supplier  will  perform  in 
his  effort  to  achieve  specified  reliability  and  maintainability. 
In  this  case  the  contractor  must  also  audit  and  control  effort 
task  performance,  involving  corrective  action  followup,  all  of 
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which  constitutes  a  much  bigger  job. 

As.-  the  contractors  design  progresses,  there  will  be  unforseen 
changes  in  the  need  for  reliability  and  maintainability,  perhaps 
because  contractor  design  improvement  tradeoffs  change  the  cri¬ 
ticality  of  the  supplier  product-.  For  example  the  contractor 
may  decide  to  add  en  alternative  mode  of  operation,  which  in  turn 
reduces  the  dependency  upon  the  suppliers  component.  Such 
changes  of  need  can  easily  change  the  suppliers  most  cost- 
effective  approach,  so  he  should  be  immediately  informed  of  them. 
A  contract  change  may  be  appropriate. 

Jut  then  there  are  inevitable  supplier  deficiencies  in  achieve¬ 
ment  of  reliability  and  maintainability  requirements,  achieve¬ 
ment  of  reliability  and  maintainability  program  tasks,  schedule 
adherence,  or  excessive  expenditure.  It  is  then  up  to  the  con¬ 
tractor  to  (a)  get  the  facts  and  thoroughly  understand  the  pro¬ 
blem,  (b)  require  official  supplier  commitment  on  corrective 
action  by  a  specified  date,  or  (c)  alter  the  requirement  on  the 
supplier  to  fit  the  unforseen  contingency,  and  (d)  possibly  alter 
the  contract. 


9. 


SUMMARY 


In  this  chapter  we  have  reviewed  the  normal  interface  between 
the  contractor  and  his  suppliers,  so  far  as  reliability  and  main¬ 
tainability  management  is  concerned.  Since  such  suppliers  pro¬ 
vide  the  bulk  of  the  hardware  that  actually  determines  the  con¬ 
tractors  achieved  reliability  and  maintainability,  careful 
management  of  suppliers  is  mandatory. 

Qualification  of  suppliers  must  be  based  upon  their  capabil  !.ty 
to  design  for  specified  reliability  and  maintainability,  not  just 
to  document  it.  The  reliability  and  maintainability  requirements 
placed  upon  suppliers  are  determined  by  (a)  the  actual  need  for 
reliability  and  maintainability,  (b)  the  technology  gap,  if  any, 
(c)  the  criticality  of  the  component  vd  thin  the  contractors 
design,  and  (d)  the  suppliers  experience  with  design  for  relia¬ 
bility  and  maintainability. 

We  have  tried  to  emphasize  that  contractor  ordering  simply  by 
supplier  product  number  is  a  dangerous  practice,  and  cannot  be 
tolerated  for  critical  compon  its.  Yet  the  contractor  should 
not-  create  new  specification  language  and  quantitative  require¬ 
ments,  which  add  to  both  contractor  and  supplier  costs,  where 
the  existing  supplier  specification  language  adequately  ties  it 
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Wo  have  also  emphasized  the  urgent  need  for  data  exchange  with 
suppliers ,  and  that  it  should  not  entail  extra  cost. 

Perhaps  most  important,  the  contractor  must  recognize  that  a 
competent  supplier  knows  far  more  his  product,  and  what 

can  be  done  to  improve  it,  than  the  contractor.  So  if  more 
reliability  and  maintainability  is  actually  needed,  the  contrac¬ 
tor  must  solicit  supplier  proposals  to  get  a  fix  on  what  is 
achievable  and  what  it  costs. 

Then  in  approaching  the  contract  negotiation  and  agreement 
phase,  and  subsequent  evaluation  and  control,  the  contractor 
should  try  for  contracts  that  require  minimum  control.  Economic 
incentives  and  penalties  surrounding  reliability  and  maintain¬ 
ability  achievement  are  powerful  medicine.  If  the  needeu 
supplier  refuses  to  commit,  himself  to  such  guarantees,  he  is 
asking  for  a  lot  of  detailed  a^d  costly  "over-the-shoulder" 
control  of  "how"  he  does  his  job. 
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Chapter  20 

MANUFACTURE  AND  OPERATION 


We  have  constantly  reiterated  the  need  to  get  the  required  re¬ 
liability  and  maintainability  into  the  design  itself,  long  before 
release  to  manufacture.  Nearly  all  other  chapters  are  addressed 
to  the  techniques  for  doing  so.  And  if  it.  is  not  done,  nothing 
we  can  do  in  manufacture  (including  "zero  defects")  01  in  opera¬ 
tion  of  the  system,  can  restore  the  lacking  inherent  reliability 
and  maintainability. 

On  the  other  hand  the  manufacturing  processes  can  and  usually  do 
degrade  the  reliability  below  that  inherent  in  the  design.  They 
reduce  the  reliability  from  a  negligible  to  a  substantial  amount 
depending  upon  (a)  the  care  with  which  the  design  allows  for 
manufacturing  state-of-the-art  capability,  or  the  design  "pro- 
ducibility",  (b)  the  adequacy  of  quality  control,  and  (c)  the 
attention  to  the  items  discussed  in  this  chapter,  which  may  or 
may  not  be  considered  quality  control  tasks. 

Quality  Control  has  been  basically  concerned  with  (a)  the  control 
of  manufacturing  processes  and  procedures  that  can  affect  product 
quality,  and  (b)  the  inspection  and  test  of  hardware  during  man¬ 
ufacture,  to  catch  and  correct  all  defects  that  car.  be  found  by 
such  methods.  The  bulk  of  the  quality  control  procedures  catch 
current  defects,  and  those,  which  "could"  lead  to  operational 
trouble,  rather  than  those  involving  reliability  or  an  operation- 
al  time  to  failure.  However  many  contractors  Quality  Control 
groups  today  do  conduct  MTBF  tests,  constituting  the  contractors 
product  reliability  "measurement"  function. 

BuShips  can  make  sure  that  the  contractor  controls  reliability 
in  manufacture  by  delineating  in  the  specification  or  program 
plar  the  tasks  given  in  chapter  22  section  16.  Methods  for  audit 
and  evaluation  of  such  contractor  effort  is  covered  in  chapter 
24  section  5,  to  be  carried  out  by  InsMat,  SupShips  or  oth^r  cog¬ 
nizant  government  representative  as  outlined  in  chapter  24  sec.  7. 

In  the  following  sections  we  will  discuss  those  activities  having 
an  impact  on  reliability  (operational  continuity  with  time) , 
excluding  those  involving  only  current  defects,  regardless  of 
the  contractors  assignment  of  responsibility. 
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SUPPLIER  CONTROL 


Supplier  reliability  and  maintainability  relationships  are  dis¬ 
cussed  in  detail  in  chapter  19.  V7e  will  only  reiterate  what  is 
said  by  its  sections  5  and  V ,  that  the  conventional  and  excellent 
supplier  quality  control  survey  techniques  seldom,  cover  the 
suppliers  design  capability.  But  once  the  design  is  released  to 
manufacture,  supplier  control  is  primarily  a  manufacturing  re¬ 
sponsibility,  thus  covered  here. 

The  suppliers  welding  capability,  of  both  machines  and  welders, 
can  have  a  significant  effect  on  reliability  depending  upon  the 
design  configuration  and  safety  margin.  In  order  to  assure 
itself  of  that  capability,  the  contractor  will  often  "certify" 
the  suppliers  welders  and  machines,  or  require  the  supplier  to 
do  so. 

Soldering  capability  has  the  same  or  higher  impact  on  electronic 
equipment  reliability  certification  of  personnel  again  being 
used. 

Radiographic,  ultrasonic,  and  infrared  inspection  techniques, 
since  they  check  the  above  fabrication  and  assembly  methods, 
also  usually  require  the  same  certification. 

Corrective  action  control  is  discussed  in  chapters  21  section  8, 
and  22  section  13.  Where  reliability  and  maintainability  can  be 
affected,  the  contractor  can  require  his  supplier  to  use  such  a 
procedure. 

Once  the  suppliers  design  is  frozen  and  accepted  by  the  contrac¬ 
tor,  absolutely  no  changes  of  design,  materials,  or  manufacturing 
processes  should  be  allowed  without  the  contractors  knowledge 
and  approval.  Many  system  failures  have  occurred  because  the 
supplier  made  a  small  change  that  in  his  honest  judgment  "obvi¬ 
ously"  improved  his  product,  but  he  could  not  for see  that  the 
contractors  special  manner  of  using  the  product  could  not  toler¬ 
ate  the  change. 

Since  the  supplier  should  have  much  better  testing  facilities 
and  knowledge  of  his  own  product,  it  is  nearly  always  preferable 
and  less  expensive  to  have  the  supplier  conduct  all  final  accep¬ 
tance,  tests,  suitably  witnessed  by  a  contractor's  representative. 
This  especially  applies  to  MTBF  tests,  because  the  supplier  can. 
get  more  product  hours  of  test  across  several  or  many  customers, 
with  much  higher  confidence  in  the  result. 
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Supplier  packinq  and  preservation  of  his  product  can  have  an 
effect  on  reliability,  as  discussed  in  chapter  18.  The  contrac¬ 
tor  must  insist  on  adequate  supplier  precautions. 

If  for  any  reason  reliability  and  maintainability  tests  are  not 
conducted  by  the  supplier ,  they  can  be  conducted  in  the  contrac¬ 
tor’s  Receiving  Test.  However  this  decision  should  be  part  of  an 
integrated  test  plan. 

Some  supplier  products  are  perishable  or  subject  to  corrosion  if 
not  properly  stored  prior  to  use  in  the  contractors  manufacture. 
This  often  amounts  to  predictable  reliability  degradation,  so 
protective  measures  are  required  both  by  supplier,  in  transit, 
and  in  the  contractors  plant. 

Supplier  cat a  reporting  is  discussed  in -chapter  1?  section  3.3, 
and  22  section  18.  The  contractor  must  require  the  supplier  to 
provide  stress  time  and  failure  data  on  all  critical  products 
(whose  failure  would  cause  the  contractor's  system  failure). 

Failure  diagnosis  must  be  required  of  the  supplier  for  every 
failure  of  a  critical  product,  as  discussed  in  chapter  16  and 
chapter  22  section  17.  It  is  usually  much  more  effective  and 
economical  for  him.  to  do  it  than  the  contractor,  and  a  good 
supplier  knows  he  needs  that  knowledge  to  survive. 

2  *  MANUFACTURING  CONTROL 

As  outlined  in  the  introduction  to  this  chapter,  we  are  attemp¬ 
ting  to  focus  on  the  activities  directly  affecting  reliability 
and  maintainability,  and  not  attempting  to  cover  ~>  1 1  quality 
control  act Lvi tie  : . 

2.1  CRITICAL  ITEM  HANDLING 


Manufacturing  reliability  control  begins  with  establishment  of 
(a)  the  "critical"  reliability  items  deserving  of  special  atten¬ 
tion,  and  (b)  the  characteristics  of  those  items  needing  special 
controls.  Special  attention  requires  physical  identification  so 
that  all  personnel  who  see  the  hardware  or  its  accompaning  paper¬ 
work  see  instantly  that  it  is  "special".  Since  critical  items 
also  deserve  special  handling,  one  commonly  used  method  is  the 
use  of  covered  tote  trays  of  a  distinctive  color  for  all  critical 
parts,  the  trays  often  containing  foam  plastic  cutouts  to  support 
the  component  properly.  hut  since  such  trays  have  also  been 
found  very  effective  for  general  material  control,  some  other 
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color  can  be  used  for  non-critical  items. 

2.2  EQUIPMENT  PROTECTION 

In  shipbuilding,  as  well  as  in  most  other  large  manufacturing 
activities,  the  contractor  normally  receives  much  Government- 
Furnished  Equipment  (GFE)  and  Contractor-Furnished  Equipment  (CFE) 
to  be  installed  in  the  system.  During  the  period  that  it  is  on 
hand,  both  awaiting  and  following  its  installation,  there  are 
many  opportunities  for  reliability  degradation  unless  preventive 
measures  are  taken.  So  that  the  Bureau  may  have  a  standard  of 
comparison  in  evaluating  contractors  equipment  protection  pro¬ 
cedures,  Figures  20-6  and  20-7  show  samples  of  excellent  proced¬ 
ures  established  by  a  major  shipbuilder. 

2.3  FABRICATION  AND  ASSEMBLY  INSPECTION 


The  kinds  of  things  that  degrade  reliability  (later  failure  in 
operation)  are  loose  or  overstressed  hardware,  insufficient 
thread  engagement,  lockwashers,  poor  solder  joints,  loose  solder, 
lack  of  surface  protection,  inadequate  clearances,  wiring  harness 
bending  stress  concentrations,  inadequate  support  (vs.  vibration 
and  shock),  improper  materials,  et<5.  Inspection  of  fabrication 
and  assemblies  is  normally  conducted  by  quality  control,  so- 
phased  that  each  job  matches  inspector  capabilities  (so  fatigue 
does  not  lead  to  poor  coverage) ,  and  so  that  areas  are  inspected 
before  being  covered  up  by  a  next  assembly  of  which  they  are  a 
part.  Obviously  encapsulation,  which  clearly  protects  reliabil¬ 
ity,  must  follow  detailed  inspection. 

2.4  NON- INSPECT ABLE  DEFECTS 

Many  defects  that  can  degrade  reliability  cannot  be  detected  by 
inspection.  These  particularly  include  the  material  processes, 
such  as  heat  treatment  (including  inadvertent),  welding,  plating, 
etching,  finishing,  casting,  etc.  They  are  usually  controlled 
by  (a)  rigid  control  of  the  procedure,  personnel,  and  equipment 
by  which  the  process  is  accomplished,  (b)  non-destructive  tests 
such  as  radiographic,  ultrasonic,  and  infrared,  and  (c)  sample 
destructive  tests  to  failure,  which  generates  confidence  only 
if  there  is  never  a  failure  at  maximum  operational  stress. 

Contamination  can  be  a  reliability-degrading  defect  impossible  to. 
inspect,  and  often  difficult  to  test  for.  It  often  results  in 
elaborate  "clean  rooms"  and  rigid  cleanliness  controls  for  both 
personnel  and  hardware. 


•  '■MIKA'  5  4-J'U 
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EQUIPMENT  PROTECTION  INSTRUCTIONS 
for  GFE  and  CFE 


Frequency 

Check 

F.qulpment 

Receipt  to 

Installation 

Nuclear  -  OFE 

Installation 

To  Delivery 

Instruction 

erreney  Cooling 
Fresh  Water  and 
Salt  Water 
Non - Regene  rat 1 ve 
(Purification) 
Regenerative 
(Purification) 


;am  Generator 


1.  Upon  receipt;  cap  and  maintain  In  accordance 
with  manufacturer's  Instructions. 

2.  When  Installed,  inspect  sines  for  corrosion; 
replace  If  corroded. 

3.  When  filled  with  water,  sample,  analyse  and 
treat  In  accordance  with  paragraph  2. 1.1.1. 

4.  Protect  on  shipboard  with  sheet  metal  covers 
In  accordance  with  paragraph  2. 1.1. 5. 


.  unless  otherwise  specified  oy  manufacturer, 
maintain  secondary  side  pressurized  to  5  pslg 
with  nitrogen. 

2.  Dry  lay  up.  Inspect  for  corrosion,  cleanli¬ 
ness  and  moisture.  Add  desiccant  If  required  by 
manufacturer's  Instructions.  Use  dynamic  dehu- 
mldlfylng  machine  when  specified  by  the  manu¬ 
facturer  (paragraph  2. 1.1. 2). 

3.  Wet  lay  up.  Check  that  vessel  is  filled  up 
Into  vents  with  water.  Sample,  analyze  and  test 
In  accordance  with  paragraph  2. 1.1.1. 


Pumrs  and 


Charging 
Core  Removal 
Reactor  Pressure- 
Fresh  Water 
Vacuum  _  ^ 

Main  CoolahF  ~  — 

Pumps  It  Motors 


IkXlIZEl 


Check  10" 

Hydraulically  Operated  5" 
Main  Coolant  Stop  14" 


1.  Rotate  pump  shaft  1  1/4  turns  manually. 

2.  Measure  and  record  Insulation  resistance. 

3.  Insure  that  permanent  or  temporary  heaters 
ure  energized,  where  applicable,  in  motors. 

4.  Protect  on  shipboard  with  sheet  metal  covers 
In  accordance  with  paragraph  2. 1.1. 5. 


nitrogen  in  accordance  with  manufacturer's 
Instructions . 

2.  Maintain  nitrogen  purge  of  1/2  to  1  cubic 
foot  per  hour,  whenever  pumps  are  dry. 

3.  When  pumps  are  dry,  manually  rotate  pump 
shaft  every  15  days.  Record  breakaway  and 
running  torques. 

4.  Wnen  pumps  are  wet  and  coolant  system  Is 
operable,  energize  pumps  for  5  seconds  every  15 

days . 

Pumps  with  stellite  thrust  shoes  do  not 
require  turning  or  purging. 


ter  inspection,  cover  and  protect  from 
damage  until  Installed  In  ship. 

2.  Protect  on  shipboard  with  sheet  metal  covers 
in  accordance  with  paragraph  2. 1.1.5. 


EQUIPMENT  PROTECTION  INSTRUCTIONS 
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2.  Place  covers  on  optical  windows. 

3.  Keep  all  personnel  cff“the  trolley  drive  tapes  and 
electrical  connectors. 
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2.5  VIBRATION  " TESTING" 

Vibration  "testing"  is  often  used  in  production.  Usually  con¬ 
ducted  as  a  much  lower  stress  level  than  ’will  be  encountered 
operationally,  it  has  quite  consistently  disclosed  loose  ’Hard¬ 
ware  or  solder  that  were  not  detected  by  very  thorough  inspection. 
And  frequently  it  turns  up  an  inadequate  support  problem  that 
design  engineers  could  not  foresee.  On  the  other  hand  some  designs 
are  necessarily  such  that  the  life  of  some  parts  is  a  direct 
function  of  the  time  they  are  under  vibrational  stress,.  If  such 
design  cannot  be  avoided  economica  Uy,  such  vibration  "testing" 
must  be  limited  in  such  a  way  that  sufficient  life  remains  when 
the  component  is  delivered  for  operation.  But  blanket  prohibi¬ 
tion  of  vibration  "testing"  may  in  effect  degrade  reliability. 

3  .  INTEGRATED  TEST 

If  every  part,  assembly,  component,  subsystem,  and  system  were 
individually  tested  tor  all  parameters  of  importance,  the  cost 
and  time  for  test  would  be  prohibitive.  Thus  most  contractors 
have  developed  what  come  to  be  called  an  integrated  test 

pla..  '  .  Its  essence  is  to  so  plan  all  testing  that  (a)  for  econ¬ 
omy  any  one  part  is  tested  only  once  or  twice,  (b)  such  testing 
is  done  soon  enough  that  there  is  time  for  correction  (including 
procurement  lead  time)  of  defects  discovered  before  the  compon¬ 
ent  is  needed  for  the  next  higher  level  assembly,  and  (c)  the 
tests  should  reproduce  true  operational  environments,  stresses, 
and  especially  interfaces  with  all  other  components. 

3.1  PERFORMANCE  TESTING 

To  do  all  of  the  above  is  impossible,  and  to  approach  the  best 
compromise  is  extremely  complex.  From  standpoints  (a)  and  ( c) 
above,  one  should  simply  test  the  finally  completed  system 
through  all  its  complete  operational  cycles  and  modes,  loaded  to 
operational  stresses  or  higher,  in  combined  temperature,  humidity, 
vibration,  shock  and  radiation  environment,  and  perhaps  corrosive 
or  abrasive  atmosphere,  altitude,  even  weightlessness. 

The  best  we  can  do  for  environment  is  to  develop  a  matrix  of 
operational  modes  vs.  environmental  contrib itors ,  and  use  engin¬ 
eering  analysis  and  judgment  to  select  the  apparently  worst  com¬ 
binations  of  environments  that  are  achievable. 

Then  there  are  many  components  of  a  system,  such  as  where  redun¬ 
dancy  is  used,  that  would  not  be  adequately  tested  by  a  complete 
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system  test-  So  we  back  off  to  a  series  of  component  tests  that 
should  be  conducted  as  soon  as  complete  for  reason  (b)  above, 
but  preferably  with  their  contiguous  components  to  detect  inter¬ 
face  problems ,  and  preferably  in  full  environnent .  Similarly 
some  components  require  prior  testing  at  the  assembly  level,  and 
some  assemblies  demand  prior  part  testing.  But  not  all.  Finally 
we  emerge  with  an  integrated  test  plan  that  constitutes  the  most 
economical  and  adequate  testing,  but  it  avoids  duplicate  testing 
of  parts  to  the  extent  feasible, 

3.2 _ RELIABILITY  TESTING 

Now  let's  turn  to  MTBF  testing,  a  primary7  kind  of  reliability 
testing.  The  principles  are  discussed  in  chapter  11.  Ideally 
we  could  run  the  completed  system  thru  its  operational  modes, 
fully  loaded,  at  full  operational  environment,  for  a  long  period 
of  time,  and  note  whether  or  how  often  it  fai 1  .  This  is  often 
done  for  systems  such  as  a  radar  or  a  wing  structure.  But  it 
can  be  impractical  for  other  systems  such  as  a  ship  or  a  launch 
vehicle  because  of  the  time  and  cost  involved,  and  impractical 
for  some  few-of-a-kmd  extremely-high-reliability  components 
such  as  microelectronic  assemblies  that  would  become  obsolete 
before  the  first  failure. 

Thus  it  has  become  logical  to  keep  records  of  the  stress  time 
and  failures  that  components  accumulate  in  engineering  prototype, 
manufacturing,  and  final  system  tests,  throughout  the  integrated 
test  plan-  When  the  system  is  shipped  to  the  user  we  then  have 
at  least  some  data  on  its  reliability,  ranging  from  "x  hours 
with  no  failures"  (for  an  MTBF  estimate  with  very  low  confidence) 
to  an  estimate  of  MTBF  with  good  confidence  if  there  were  a 
dozen  failures.  Sometimes  this  technique  is  supplemented  by  a 
relatively  short  MTBF  test  as  outlined  above,  to  obtain  reason¬ 
able  confidence  that  the  reliability  is  adequate. 

On  the  other  hand  MTBF  testing  is  impractical  for  some  systems 
and  components,  particularly  structures,  because  failures  so 
rarely  occur.  As  discussed  in  chapters  7,  11,  and  13,  the  only 
feasible  "reliability’'  test  is  one  or  more  overstress  tests  to 
failure.  Such  tests  to  failure  are  normally  accomplished  during 
the  design  phase,  as  such  a  test  after  manufacture  would  normal¬ 
ly  be  too  late  to  have  an  impact  on  the  design. 

However  many  ov  rstress  tests  not  to  failure,  such  as  1.5-times 
operating  pressure  tests,  provide  considerable  confidence  in  the 
design  strength  and  quality. 


20-10 


3.3  MAINTAINABILITY  TESTING 

Maintainability  tests  do  net  have  the  tine  and  cost  problem 
associated  with  reliability  tests,  so  can  nearly  always  be  con¬ 
ducted.  But  the  planning  and  desigr  of  such  tests  can  be  qux^e 
complex.  Again  it  is  desirable  to  conduct  them  on  the  complete 
system,  this  time  surrounded  by  a  mockup  of  its  operational 
location.  Most-likely  failures  are  simulated  and  technicians 
(of  skill  and  training  matching  the  users  technicians)  timed  to 
determine  how  fast  they  can  find  and  fix  the  failure.  But 
again  some  maintainability  tests  have  to  be  conducted  at  lower 
component  levels  so  that  there  will  be  time  to  make  corrections 
as  a  result. 

3.4  FAILURE  DIAGNOSIS 

At  the  risk  of  seeming  to  emphasize  the  point,  since  some  con¬ 
tracts  actually  limit  the  number  of  diagnoses,  absolutely  every 
failure  occurring  during  test  must  be  diagnosed.  Whenever  the 
cause  cannot  be  determined,  the  component  is  forever  suspect. 
Many  contractors  follow  a  policy  of  scrapping  every  such  com¬ 
ponent  that  is  in  a  critical  application. 


4 *  DELIVERY  AND  INSTALLATION 

4.1  PACKING  AND  PRESERVATION 


Just  as  in  section  1  above  we  were  concerned  about  the  supplier's 
packaging  and  preservation  of  completed  systems  and  components 
sent  co  the  contractor,  the  contractor  must  be  at  least  equally 
careful  about  packing  and  preservation  of  completed  systems  and 
components  sent  to  the  installation  site. 

Many  design  engineers  fail  to  realize  th-t  the  temperature 
humidity,  vibration,  and  especially  shock  encountered  in  a 
cros8-country  van  or  freight  often  far  exceed  the  operational 
values  to  which  he  designed.  But  if  it  is  not  designed  for 
these  values,  and  sometimes  it  cannot  bo,  i*  is  up  to  the  pack¬ 
ing  expert  to  install  whatever  protection  and  supports  are 
needed.  Failure  to  do  so  degrades  reliability  by  overstressing 
components  "almost"  to  failure,  so  that  the  later  operational 
stress  may  complete  the  failure.  And  it  is  up  to  the  design 
engineer  to  advise  the  packing  expert  about  sensitive  components. 

When  the  system  arrives  at  its  operational  site,  there  must  be 
clear  unpacking  directions  to  avoid  inadvertent  damage.  If  the 
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damage  ls  not  visible  (say  a  wire  a ’most  broken  within  unbroken 
insulation)  the  reliability  has  been  degraded. 

4.2  INSTALLATION 

Installation  involves  the  same  hazards  to  reliability  as  Packing 
and  Preservation,  in  that  very  careful  handling  of  many  components 
may  be  required  to  prevent  overstress  and  there  tore  hidden  unre¬ 
liability.  Such  equipment  should  be  clearly  labelled  to  attract 
attention,  with  handling  instructions.  Such  procedures  are  nor 
new,  but  consideration  of  tine  reliability  (,rs  .  operational  time) 
implications  will  often  result  in  additional  handling  and  instal¬ 
lation  constraints. 

4.3  CHECKOUT 


Checkout  is  undertaken  when  installation  is  physically  complete, 
normally  done  by  technicians  to  instruction  manuals,  under  the 
surveillance  of  one  or  more  engineers  very  familiar  wi vh  the 
system.  Checkout  offers  many  opportunities  for  reliability  de¬ 
gradation.  When  the  technicians  screwdriver  slips  and  rams  into  a 
mechanism,  he  may  inspect  it  carefully  and  try  working  the  mech¬ 
anism.  If  it  seems  still  to  work  (within  his  own  framework  of 
knowledge)  he  considers  that  the  end  of  it  .  But  it  may  have  been 
overstressed,  almost  broken,  misaligned ,  etc.  in  a  way  that  ho 
cannot  see,  and  later  fails  when  needed  operationally.  This  is 
not  uncommon.  Tt  happens  all  the  time-.  And  there  is  also  the 
kind  of  man  who  knows  he  damaged  it  but  does  not  report  it.  Of 
course  the  design  should  'ovo  been  " foolproof"  in  the  first  place. 
Thus  there  must  be  very  t.  uorough  indoctr  inat  ion  and  training  of 
technicians,  emphasizing  the  consequences  of  any  failure  to  re¬ 
port  posr ible  overstress,  and  there  should  be  no  punitive  action 
for  such  reporting. 


c.  OP  HR AT  ION  AMD  MA I NT ENANC  E 

The  opportunities  for  degradat ion  of  reliability  and  maintainabi 1- 
ity  in  operation  and  maintenance  are  enormous.  The  largest  single 
cause  of  unreliability  in  operational  systems  is  probably  not  the 
hardware  at  all,  but  human  error.  For  some  systems,  such  as  large 
computer  installations,  it  has  been  estimated  that  4t>.v  of  the 
downtime  is  due  to  human  mistakes,  although  many  of  these  are 
charged  to  hardware  failure.  People  are  well-intentioned,  often 
careful  and  dedicated,  but  fallible.  What  can  we  do  about  it? 

As  discussed  in  chapter  14  we  must  use  people  only  for  functions 
they  can  per  form  better  than  hardware,  then  design  the  har  dware 
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for  compatibility  with  human  capabilities  and  frailties.  Then  we 
have  to  thorouqhiy  train  and  motivate  them,  showing  the  conse¬ 
quence  of  error. 

5 « 3.  SPARES 

Spare  components  and  their  logistics  of  supply  also  have  an 
impact  on  reliability  and  maintainability.  Spares  needed  for 
preventive  maintenance  obviously  degrade  reliability  if  they  are 
not  available  when  needed.  Spares  needed  for  corrective  mainten¬ 
ance  after  a  failure  obviously  degrade  operational  maintainability 
(but  not  hardware  maintainability  by  the  current  evolutionary 
definition)  if  they  are  not  available  to  restore  the  system. 

Lack  of  either  of  course  degrades  system  availability  and  effec¬ 
tiveness.  The  only  solution  to  this  is  of  course  a  complete 
maintenance  policy  a. id  logistics  program,  which  are  beyond  the 
scope  of  this  course.  The  Rureau  of  Supplies  and  Accounts  is 
vigorously  attacking  this  p;  colem  under  the  METRI  program. 

5 .2  "ABNORMAL"  CONTRITIONS 

Systems  are  designed  to  operate  within  specified  environmental 
and  stress  ranges,  to  utilize  specified  inputs,  and  to  perform 
specified  functions.  But  such  specifications  cannot  always  en¬ 
compass  what  actually'  happens.  Unforseen  environments  ana  stresses 
are  sometin.es  applied,  the  inputs  are  sometimes  far  out  of  toler¬ 
ance,  i r  an  attempt  is  made  to  perform  a  fur 't ion  beyond  capabil¬ 
ity  of  the  system.  Such  abnormal  conditions  are  often  not  appar¬ 
ent  in  the  Paperwork  when  a  "failed  part"  gets  back  to  the  tender 
or  manufacturer.  So  the  system  contractor  gets  a  "black  eye" 
that  he  may'  not  deserve.  But  the  system  itself  is  indeed  unrelia- 
ole  in  the  real  environment  that  was  abnormal  only  in  relation  to 
an  inadequate  speci fi cat  ion.  Therefore  it  is  important  to  make 
every  effort  to  get  the  r»  a  1  situation  into  the  original  design 
requirements,  and  to  report  the  rea 1  situation  surrounding  each 
fai lure . 

5.3  MAINTENANCE 

However,  it  is  in  maintenance  that  the  primary  operational  unre¬ 
liability  is  generated,  and  there  are  some  unresolved  problems. 
First  there  is  the  manufacturers  rccotnmendo  1  preventive  mainten¬ 
ance  schedule,  which  he  has  careful lv  worked  out  to  achieve  opti¬ 
mum  reliability.  Perhaps  more  often  than  not,  such  schedules  are 
not  followed  by  the  user.  The  result  is  less  reliability  than 
that  of  which  tne  component  and  system  are  capable.  We  can  call 
this  part  ol  the  human  error,  since  the  hardware  is  rtamly 
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per  form d  rtg  to  specification.  Therefore  r.'rte  user  should  have  man¬ 
datory  maintenance  procedures  that  require  adherence  to  the  manu¬ 
facturers  plan  (as  specified  in  the  Navy  manuals),  and  either  cn- 
iorce  the  procedures  or  depart  from  them  only  with  conscious 
knowledge  of  the  reliability  consequences. 

Second  is  the  corrective  maintenance,  involving  detection  of  mal¬ 
function,  diagnosis,  acquisition  of  spares,  rep’ acement  or  repair, 
checkout,  and  sometimes  reload.  Detection  and  diagnosis,  and  to 
a  lesser  extent  repair  and  checkout,  require  certain  skills  that 
may  or  may  not  be  available.  Of  course  the  hardware  should  be 
designed  for  maintenance  with  specified  skill  levels,  but  if  the 
manpower  and  skill  levels  actually  available  are  something  less 
than  specified,  which  often  occurs,  maintainability  and  therefore 
avail  hility  is  degraded.  Therefore  the  equipment  must  be  designed 
for  the  real  skill  level  available,  and  maintenance  personnel 
thoroughly  trained. 

Third,  and  applicable  to  both  preventive  and  corrective  mainten¬ 
ance,  is  the  reliability  degradation  due  to  maintenance  errors. 

As  discussed  in  section  4  above,  when  t no  technicians  screwdriver 
slips  and  damages  a  mechan'sm  (but  he  does  not  think  it  is  dam¬ 
aged)  ,  or  he  accidentally  shorts  two  electrical  terminal-.'  which 
damages  a  brans  is tor ,  or  ho  plugs  a  connector  into  the  wrong 
socket,  or  he  uses  the  wrong  replacement  part,  etc.,  --  all  these 
fact-of-life  occurrences  do  degrade  reliability.  And  often  very 
substantially.  As  stated  in  4.3  above,  there  must  be  very  thorough 
indoctrination  and  training  >  t  technicians,  emphasizing  the  con¬ 
sequences  of  any  failure  to  report,  and  fix  possible  damage,  and 
there  should  be  no  punitive  action  tor  so  report ing  * 

5.4  DATA  R  ECORP  INC. 

Those  human  errors  are  very  difficult  to  record,  because  o  f  human 
reluctance  to  admit  mistakes,  and  therefore  difficult  to  get  back 
to  the  design  engineer  so  he  can  make  the  next  design  more  tool- 
proof.  But  it  is  a  1 nost  as  difficult,  in  spite  of  complete  data 
collection  systems,  to  get  good  operating  stress  time  and  failure 
data  on  truly  defective  failed  parts.  In  both  military  and  in¬ 
dustrial  systems  (such  as  large  Leased  computer  systems  where 
maintenance  cost  comes  out  of  the  manufacturer's  pocket ) ,  it  has 
been  found  that  the  maintenance  technician  often  cannot  effectiv¬ 
ely  collect  the  required  data.  Ho  is  highly  motivated,  anu 
properly,  to  'get  the  system  going1'  again  as  fast  as  he  can.  In 
industry  his  pay  raises  may  depend  on  this  capability  relative  to 
his  contemporaries. 
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On  hipboarci,  during  active  use  of  the1  equipment,  the  same  urgency 
obtains.  Technicians  arc  ny  regulation  required  ro  fill  out  the 
Electronic  Equipment  Failure  Report  on  absolutely  e\r,ry  electronic 
equipment  failure.  But  (a)  he  is  usually  unable  to  record  the 
stress  time  or  cycles,  but  is  asked  to  "estimate"  them,  (bi  he 
seldom  records  human  failure  when  it  is  tn._  actual  cause,  (c)  he 
often  fails  to  recognize  a  failure  as  a eh,  so  it  is  not  recorded, 
and  (d)  in  times  of  urgency  he  may  have  to  defer  reporting  until 
he  has  to  guess  what  happened. 

As  a  result,  data  reporting  has  been  far  from  adequate  for  the 
design  engineer  to  identify  the  needed  design  corrections.  Non¬ 
electronic  equipment  failure  reportinc  appears  to  be  even  less 
adequate . 

The  Navy  Maintenance  Management  System,  (see  chapter  9)  has  been 
developed  and  it  should  improve  the  situation,  but  further  im¬ 
provement  is  needed  to  incorporate  human  failure  data. 

6.  SUMMARY 

In  this  chapter  we  have  briefly  t.  is  cussed  the  primary  activities 
for  reliability  control,  rather  chan  "quality  centre. 1",  dur  ing 
the  manufacturing  phase.  These  are  primarily  supplier  control, 
critical  item  handling,  control  of  non- inspect  able  detects,  vi¬ 
bration  testing,  integrated  testing  for  reliability  and  maintain¬ 
ability,  failure  diagnosis ,  and  careful  packing  and  preservation. 

Then  we  have  discussed  the  practices  during  installation  and 
checkout,  f  o  1 1  owed  by  the  activities  during  actual  oper.it  ion  and 
maintenance.  In  this  operational  phase  the  primary  factors 
affect ing  reliability  and  maintainability  are  spares,  abnormal 
conditions,  quality  ot  preventive  an  i  correct  ive  maintenance,  and 
dat a  record. in-; . 

We  em.phus  ire  that  a  v  ry  substantial  put  of  system  unreliability 
is  caused  ict.  by  the  hardware  itself,  but  by  the  way  t  is  used 
and  mair.t  a  l.  tied. 
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Chapter  21 

CONTRACTOR  ORGAN 17 AT  ION 

If  the  contractor  produces  what  he  contracts  to  produce,  on  time, 
and  within  cost,  BuShips  personnel  need  not  be  concerned  with 
how  the  contractor  organizes  to  do  the  job.  Of  course  this  is  a 
"big  if".  But  if  BuShips  cannot  be  sure  the  contractor  will  do, 
or  knows  how  to  do,  his  job,  there  is  a  temptation  to  tell  him 
how.  And  the  contractor,  needing  the  contract,  may  accept  such 
direction.  Chapter  23  section  3.4  quotes  the  MIL  STD  785  guid¬ 
ance  . 

But  if  BuShips  tells  him  how  to  organize,  BuShips  is  substituting 
its  own  judgment  (remote  from  the  problems  of  the  contractor)  f^r 
the  judgment  of  the  contractor,  who  is  in  a  much  better  position 
to  understand  the  resources  ano  constraints  unique  to  his  opera¬ 
tion.  So  it  is  always  preferable,  when  feasible,  to  negotiate  a 
contract  with  sufficient  incentive  that  the  contractor  will  be 
willing  and  able  to  organize  properly'  to  do  the  30b.  He  may'  need 
suggestions  from  BuShips,  but  the  decision  has  to  be  the  contrac¬ 
tors  . 

There  are  almost  as  many  kinds  of  organization  as  there  are  con¬ 
tractors,  and  the  same  is  largely  true  of  their  reliability  and 
maintainability  activities.  Yet  there  are  some  fairly  common 
patterns .  In  this  chapter  we  will  try  to  show  the  more  "typical" 
patterns  of  organization  for  reliability  and  maintainability, 
but  we  cannot  say  that  "most"  contractors  use  any  one  pattern. 

We  are  using  the  word  "organization"  in  a  broad  sense,  to  mean 
how  he  arranges  resources  and  work  flow  to  get  the  job  done, 
rather  than  the  narrower  "organization  chart"  sense. 


1  *  WORK  FLOW 

Referring  tc  figure  21-3,  let  us  review  the  sequence  for  a  large 
project.  While  this  total  worK  flow  is  given  for  illustration 
of  the  scope,  most  of  the  steps  are  required  to  some  degree  in 
i._arly  every  contract.  For  example  procurement  of  an  off-the- 
shelf  hardware  item  would  not  involve  an  RFP  or  system  design. 
This  chart  does  not  imply  any  particular  organization  structure 
or  jurisdiction  of  Reliability  and  Maintainability  (R&M)  groups. 


The  Request  for  Proposal  (RFP)  for  a  large  project  involving 
system  design  is  thoroughly  analyzed  by  an  engineering  group 
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(or  an  advanced  systems  group)  to  examine  the  requirements  and 
leas ioi li Ly ,  feasibility  can  only  be  evaluated  by  considering 
alternative  preliminary  designs  and  selecting  one  that  will  be 
proposed.  The  contractors  R&M  spe  ralists  must  provide  (a)  i_he 
necessary  tools  and  techniques  tor  analysis  that  accounts  for 
reliability  and  maintainability,  (b)  the  tools  and  techniques  to 
design  for  required  reliability  and  maintainability,  and  (c)  may 
perform  the  analyses  if  they  require  techniques  in  which  only 
they  are  experienced. 

When  the  proposed  design  is  established,  a  proposal  is  developed. 
The  R&M  Program  Plan  within  the  proposal  is  developed  by  the  R&M 
specialists,  working  very  closely  with  the  design  specialists. 
After  BuShips  evaluation  and  selection  of  a  contractor,  a  con¬ 
tract  is  negotiated  containing  the  negotiated  R&M  Proqram  Plan. 

Full  system  design  then  gets  under  way,  using  the  tools  and 
analyses  provided  by  the  R&M  specialists,  resulting  in  block 
diagrams,  schematics,  analytical  models,  and  the  detailed  speci¬ 
fications  that  constitute  the  hardware  design  requirements. 
Detailed  hardware  design  proceeds  in  the  same  way,  this  time 
involving  evaluation  tests  of  selected  components  or  "bread¬ 
boards",  which  tests  require  R&M  specialist  analysis.  As  design 
problems  are  resolved  and  component  designs  completed,  detailed 
procurement  specifications  are  written,  and  the  R&M  specialists 
may  write  the  R&M  requirements  thereof.  Component  designs  are 
"released"  to  manufacture  as  fast  as  they  are  completed. 

Purchase  orders  on  contracts,  including  the  reliability  and 
maintainability  requirements,  are  issued  to  suppliers.  As 
suppliers  complete  each  major  item,  the  contractor  audits  his 
final  test  thereof  and  accepts  it  for  delivery.  If  for  any 
reason  it  is  not  fully  tested  at  the  suppliers  plant,  or  where 
components  from  different  suppliers  must  be  tested  together,  the 
contractor  tests  upon  receipt.  Fabrication  and  assembly  then 
proceeds  thru  subassemblies,  minor  components,  major  components, 
and  subsystems,  testing  as  required  along  the  way.  As  tests 
encounter  problems,  corrective  action  is  executed,  followed  by 
retest  of  affected  areas. 

Finally  the  contractor  determines  to  his  own  satisfaction  that 
'■11  specifications  are  met,  and  "accepts"  it.  The  SupShips  or 
Insmat  representative,  having  audited  the  final  inspection  and 
test,  accepts  if  he  concurs.  Then  it  is  shipped  for  installation 
and  operation. 

Now  let  us  refer  back  to  the  three  levels  of  engineering  design 
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activities  (Preliminary  design,  system  design,  hardware  design). 
What ,  exactly,  goes  on  in  these  blocks,  so  far  as  reliability 
and  maintainability  are  concerned?  The  design  cycle  is  shown  in 
figure  21-6  to  emphasize  the  highly  iterative  nature  of  design. 

The  design  engineer  must  satisfy  a  surprisingly  large  number  of 
requirements,  including  the  R&M  requirements,  and  do  it  within  a 
surprising  number  of  constraints,  such  as  size  and  cost.  To  the 
extent  he  can  calculate  what  the  design  should  be,  he  is  ahead. 
But  in  complex  systems  such  pre-analysis  bogs  down  in  inability 
to  think  of,  let  along  account  for,  all  interfaces.  So  he  re¬ 
sorts  to  trial  and  error  on  paper.  Over  and  over,  dozens  or 
hundreds  of  times,  until  finally  he  has  something  that  should  do 
it.  He  uses  analysis  after  each  trial  to  evaluate  whether  his 
design  meets  all  requirements  and,  if  not,  to  point  the  way  to  a 
better  solution. 

So  it  is  with  reliability  and  maintainability.  Starting  with  the 
R&M  requirements,  he  first  selects  a  design  that  he  thinks  will 
satisfy  them,  as  well  as  all  other  requirements.  Then  he  (or 
his  R&M  specialist)  evaluates  the  design  using  the  reliability 
apportionment/prediction  techniques.  If  it  is  "not  good"  he 
uses  the  analysis  to  guide  his  next  trial  design.  Then  he  goes 
around  again  until  finally,  perhaps  weeks  later,  he  works  out  a 
design  that  is  just  right"  and  can  be  released  to  manufacture. 

Now  the  point  of  all  this  is  that,  R&M  achievement  in  design  is 
the  objective.  This  entire  course  has  no  other  purpose.  In 
order  to  achieve  it,  design  evaluation  (or  analysis)  is  vital, 
but  does  not  do  any  good  by  itself.  Thus  if  the  selected  con¬ 
tractor  has  not  organized  in  such  a  way  that  the  design  engineer 
achieves  the  required  reliability  and  maintainability,  then  he 
does  indeed  need  help. 


2.  ORGANIZATION  STRUCTURE 

2.1  FUNCTIONAL  ORGANIZATION 

Today  we  call  the  traditional  organization  structure  a  "function¬ 
al"  organization,  meaning  that  the  marketing,  engineering,  manu¬ 
facturing,  financial,  etc.  "functions"  report  to  the  top  executive, 
and  that  all  personnel  report  to  one  of  these  functional  managers 
(or  vice-president,  director,  etc.)  according  to  their  skills. 

It  is  a  grouping  by  skills  rather  than  projects.  Functional 
organizations  have  the  advantage  of  smoothing  manpower  require¬ 
ments  for  each  kind  of  skill,  and  therefore  building  and  conser- 
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ving  knowledge  and  experience  in  each  technology  required,  and 
j  profiting  from  cross- fertilization  of  ideas  between  experts. 

Figure  21-8  shows  the  more  common  structural  location  of  R&M 
groups  in  a  functional  organization.  Pernaps  the  commonest  and 
a  very  effective  approach  is  £»n  R&M  group  (by  whatever  name) 
reporting  to  the  Engineering  manager,  giving  it  equal  stature 
with  all  other  engineering  management  activities.  If  Engineer¬ 
ing  then  has  major  project  groups,  such  as  "Project  A",  the 
latter  may  either  use  the  services  of  the  top  R&M  group,  or  may 
in  addition  have  its  own  R&M  specialists  or  group.  But  if  there 
is  no  policy-setting  and  technique-generating  top  engineering 
R&M  group,  one  might  question  the  effectiveness  of  the  lower 
level  project  group. 

Many  companies  have  established  Product  Assurance  groups  (also 
called  by  other  names),  reporting  to  the  top  executive,  which 
essentially  combine  Quality  Control  and  Reliability  (rarely 
maintainability).  This  is  an  outgrowth  of  the  Q.C.  philosophy 
that  reliability  is  "controlled"  more  than  it  is  achieved  by 
design.  As  a  result,  such  groups  commonly  extrapolate  the 
excellent  quality  control  doctrines,  techniques,  viewpoints,  and 
disciplines  into  design. 


But  this  arrangement  can  work  very  well  if  the  Engineering  and 
Product  Assurance  managers  actually  understand  the  problem  and 
actually  support  the  R&M  group  technical  activities.  All  too 
often  this  does  not  occur,  and  the  design  engineers  have  as  little 
communication  as  possible  with  the  R&M  engineers. 

Sometimes  the  R&M  group  is  placed  at  the  next  lower  echelon  with¬ 
in  Product  Assurance,  which  makes  it  work  "uphill"  to  the  major 
Engineering  groups.  This  does  not  work,  because  the  R&M  group 
is  unable  to  get  the  "ear"  of  the  principal  engineering  functions. 
Most  experienced  and  competent  R&M  people  know  this  situation 
only  too  well,  so  will  not  accept  or  stay  in  positions  so 
structured . 

Some  contractors  have  tried  to  organize  R&M  as  part  of  Quality 
Control,  which  demonstrates  to  knowledgable  customers  their  lack 
of  understanding  of  the  meaning  of  reliability  and  maintainability 
achievement.  The  reason  is  that  Quality  Control  commonly  has 
many  people  labelled  "engineers”  who  are  not  engineers  at  all, 
so  competent  desiqn  or  R&M  engineers  fear  associative  damage  to 
their  professional  stature  and  probably  their  salary.  For  the 
same  reason  the  design  engineering  groups  understandably  resist 
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"control"  (the  QC  philosophy)  by  people  they  consider  unable  to 
understand  the  design  problem.  Now  this  is  a  rather  drastic 
simplification,  but  these  situations  do  in  fact  adversely  affect 
achievable  reliability  and  maintainability. 

2.2  PROJECT  ORGANIZATION 


One  great  disadvantage  of  at  least  a  large  functional  organiza¬ 
tion  structure  is  the  tendency  for  any  one  project  to  get  lost 
in  it.  There  is  little  management  of  the  project  per  se,  sched¬ 
ules  slip  seriously  before  anyone  is  aware  of  it,  and  costs 
begin  to  rise.  One  solution  to  this  problem  is  to  have  only 
projects  reporting  to  the  top  executive,  each  project  having  its 
own  independent  functional  organization.  This  does  indeed 
manage  each  project  much  better,  but  results  in  little  or  no 
communication  of  technology  across  projects,  gross  duplication 
of  effort,  and  very  difficult  manpower  loading  and  turnover. 

2.3  BALANCED  ORGANIZATION 


In  attempts  to  find  the  best  balance  between  the  above  two 
extremes,  the  "matrix",  "hybrid"  and  many  other  schemes  have 
been  used.  But  the  principal  trend  seems  to  be  toward  the 
balanced  organization  shown  in  figure  21-10.  The  general  scheme 
is  that  both  functional  and  project  managers  report  to  the  top 
executive,  which  makes  quite  a  management  span  for  him. 

Taking  all  engineering  employees,  for  example,  the  central 
Engineering  department  is  their  "home”,  and  responsible  for 
hiring,  salaries,  and  firing.  The  central  Engineering  organiza¬ 
tion  maintains  long-term  centers  of  technology,  such  as  RAM, 
where  outstanding  experts  are  located  more  or  less  permanently 
to  develop  techniques  and  provide  consultation.  Such  an  RAM 
group  takes  care  of  all  RAM  analysis  and  programs  for  new  busi¬ 
ness  . 


When  a  project  is  formed  as  the  result  of  a  contract,  the  nec¬ 
essary  engineering  people  transfer  to  the  project  for  its  dura¬ 
tion,  including  RAM  specialists.  But  the  RAM  policy  and 
technique  generation  remains  with  the  central  organization,  as 
well  as  the  arrangement  of  regular  meetings  where  RAM  specialists 
keep  each  other  up  to  date  on  techniques.  At  the  conclusion  of 
the  project  they  are  transferred  back  to  their  hone  group  to 
work  between  projects. 


In  such  a  balanced  organization  the  RAM  group  most  commonly 
reports  to  the  central  Engineering  manager  as  shown.  Or  it  may 
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be  called  by  broader  names,  such  as  Product  Ef fectiv<_resc  ,  System 
Effectiveness,  or  Design  Assurance.  As  discussed  in  2.1  above,  it 
would  be  ineffective  operating  "up:..  11"  from  a  lower  echelon. 

"Project  A"  would  have  it.  own  R&M  (or  another  name)  group  re¬ 
porting  either  to  the  Project  manager  (if  R&M  is  a  quite  major 
consideration)  nr  to  the  Project  engineering  manager  in  the 
commoner  case.  It  then  is  on  the  same  level  with  all  major  en¬ 
gineering  groups  of  the  project,  which  enables  it  to  work  well. 

Alternatively  the  central  R&M  function  may  report  to  the  central 

Product  Assurance  function  if  there  is  one.  This  is  an  uncommon 

arrangement,  but  can  work  if  che  Product  Assurance,  Engineering, 
Project  A,  and  Project  A  Engineering  managers  all  understand  the 
problem  and  support  the  central  R&M  technical  effort. 

Again,  as  above,  putting  R&M  in  a  lower  tier  or  reporting  to 
Quality  Control  does  not  w'ork . 

2.4  WHY  SPECIALIZED  GROUPS? 


Many  thoughtful  managers  question  the  need  for  special  Reliabil¬ 
ity  and  Maintainability  groups  of  specialists,  and  it's  a  legi¬ 
timate  question.  If  reliability  and  maintainability  can  be 
achieved  only  by  the  design  engineer,  then  let's  te^ch  him  how, 
and  why  do  we  need  the  specialists? 

One  reason  is  that  the  reliability  and  maintainability  technolo¬ 
gies  are  new,  very  fast-growing,  generating  a  fantastic  number 
of  technical  articles.  If  the  design  engineer  tried  to  read  all 
this  material,  to  sift  out  what  he  needs  or  can  use,  he  would 
have  mo  time  left  to  design  anything.  The  specialist  has  to 
cover  it,  extract  what  is  useful,  then  either  use  it  or  convert 
it  to  concise  form  for  the  design  engineers. 

Another  reason  is  th^t  although  many  of  the  techniques  are  well- 
developed,  others  are  not,  but  urgently  needed,  structural 
reliability  prediction  and  hardware  cost-e f feet iveness  analysis 
are  examples.  The  design  engineer  can  seldom  take  time  out  to 
develop  a  new  teennique. 

Another  is  that  reliability  and  maintainability  design  and 
analysis  is  a  small  part  of  the  design  job.  Many  design  groups 
could  not  iustify  a  full-time  reliability  and  maintainability 
specialist.  The  need  for  reliability  and  maintainability  anal¬ 
ysis  is  sporadic.  In  the  interest  of  efficiency  and  technolo¬ 
gical  continuity  it  makes  more  sense  to  establish  a  central 
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"ban);"  of  such  capability  upon  which  the  design  engineer  can 
draw  as  he  needs  it  (or  upon  which  management  can  draw  for  audit). 
The  widespread  practice  of  establishing  "stress  analysis"  groups 
is  directly  analogous. 

And  last  but  not  least  the  establishment  of  a  reliability  and 
maintainability  specialists  group  is  probably  the  most  effective 
way  to  get  initial  attention  and  action,  as  well  as  continuous 
emphasis.  They  won't  let  the  design  engineer  overlook  reliabil¬ 
ity  and  maintainability  because  of  other  pressures. 

3.  POLICY  and  PROCEDURE 


A  good- loo1  ing  organization  structure  is  worthless  until  there 
is  clear  and  concise  direction  from  management  stating  (a)  what 
is  to  be  done,  (b)  exactly  who  has  the  respons ibi 1 itv  and  author¬ 
ity  to  do  it,  and  (c)  how  it  is  enforced. 

Such  policy  direction  concerning  reliability  and  maintainaoi lity 
is  usually  issued  by  the  top  executive,  by  the  Engineering  execu¬ 
tive,  or  by  the  Product  Assurance  executive.  It  makes  little 
difference  who  issues  it  so  long  as  all  three  really  come  to 
agreement  and  understanding  on  the  subject. 

Detailed  interpretation  of  Policy  is  usually  accompli  ■'-bed  by 
Procedures  written  by  the  departments  primarily  concerned,  care¬ 
fully  obtaining  agreement  of  the  affected  departments.  These 
include  specific  work  flow  between  named  departments,  stating 
which  department  has  responsibility  and  authority  for  what,  and 
what  standard  documents  are  to  be  used. 

There  are  some  chronic  problems  with  both  policies  and  procedures 
however,  that  may  be  of  concern  to  BuShips  when  there  is  trouble 
with  a  contractor.  Perhaps  the  worst  offender  is  their  issuance 
in  such  weasel-worded  form  that  responsibility  and  authority  are 
not  actually  pinned  down.  Sometimes  this  r<  fleets  )  onest  differ¬ 
ence  of  intei pretat ion  of  the  words,  but  all  too  often  it  re¬ 
flects  knowing  compromise  to  the  extent  that  the  Policy  or  Pro¬ 
cedure  is  meaningless. 

Another  problem  is  that  while  the  document  may  be  adequately 
concise,  it  is  either  not  kept  up  to  date  or  not  enforced.  Com¬ 
petent  managements  have  established  periodic  audits  to  discover 
such  discrepancies,  and  to  either  correct  the  document  or 
arrange  enforcement 
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Another  serious  problem  is  the  tremendous  flow  of  specifications 
out  of  government  agencies,  many  of  which  conflict  with  each 
other.  Each  agency's  representative  has  his  own  interpretation, 
leading  to  weasel-worded  procedures  to  satisfy  several  conflic¬ 
ting  masters  . 

Reliability  and  maintainability  policies  and  procedures  ; -e 
particularly  vulnerable  to  these  problems  because  of  the  newness 
of  the  techniques  to  more  contractors.  If  BuShips  will  state 
exactly  what  reliability  and  maintainability  is  required 
contractually,  particularly  if  it  is  fixed  price  incentive, 
contractor  management  will  quickly  face  the  reality  and  decide 
exactly  who  does  waat  to  Willie. 

4.  RESPONSIBILITY  ASSIGNMENT 

Having  established  the  importance  of  contractor  assignment  of 
responsibility,  where  should  he  assign  it?  Many  of  the  tasks 
that  have  an  important  impact  on  reliability  and  maintainability, 
and  which  are  called  out  in  reliability  and  maintainability 
specs,  do  not  belong  in  a  Reliability  and  Maintainability  group 
because  of  their  much  broader  nature.  Parts  control  is  one  of 
these.  Design  review  is  another,  although  often  conducted  by 
the  Reliability  and  Maintainability  group. 

Figure  21-14  shows  a  fairly  typical  assignment  of  task 
responsibilities  (a)  between  centralized  functions  and  projects 
and  ( b )  between  design  and  Reliability  and  Maintainability 
engineering  groups.  Any  one  line  across  the  chart  represents 
a  single  principal  task,  as  outlined  in  Chapter  22,  but  with 
modifiers  in  each  column  to  indicate  roughly  what  part  of  each 
is  done  where. 

But  again  if  there  is  contractor-wide  recognition  of  exactly  who 
is  responsible  for  each  task,  and  there  is  competent  management, 
it  makes  little  difference  where  it  is  done. 

To  go  a  step  deeper,  Figures  21-16  and  21-16  from  (1)  NAVSHIPS 
94501  show  a  typical  listing  of  tasks  on  the  piesumption  that 
there  will  be  Reliability  Analysis,  System  Reliability,  Parts 
Reliability,  and  Reliability  Test  groups.  While  this  is  an 
excellent  organization  (Federal  Electric  Cot  p) .  it  is  only  one 
contractor.  Most  other  contractors  would  ha’-  «*  to  do  something 
differently  to  fit  their  own  structures. 


TYPICAL  RESPONSIBILITY  ASSIGNMENT 
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5. 


EDUCATION  AND  MANUALS 


As  recognized  by  BuShips  in  its  Procurement  Request  for  this 
course,  reliability  and  maintainability  education  has  to  begin 


worP  allocated  for  renabili 
management  can  see  why  it's 
with  the  need  to  "do  someth! 
ability,  management  will  hir 
"expert"  and  expect  him  to  " 
of  course.  It  doesn't  work, 
educating  management.  And  1 
Part  o:;  management's  job  is 
convinced  of  the  reason  for 


will  be  spent,  not  an  hour  of 
ty  and  maintainability  effort,  until 
necessary.  All  too  frequently,  faced 
ng"  about  reliability  and  maintain- 
e  a  reliability  and  maintainability 
take  care  of  it."  Without  any  cost, 
unless  the  expert  somehow  bear-ins  by 
et's  "Ot  be  too  hard  on  management, 
to  avoid  spending  money  until 
it . 


Then  comes  Engineering,  and  we  mean  the  people  who  invent, 
create,  and  design  the  system,  subsystems,  and  components  to  be 
manufactured  or  procured.  until  all  the  contractors  design 
engineers  have  some  knowledge  of  reliability  and  maintainability 
achievement  and  analysis  techniques,  and  some  design  engineers 
become  real  experts,  management  can  hardly  expect  achievement  in 
the  design  itself.  And  there  is  no  other  place  to  get 
reliability  and  maintainability. 

One  solution  to  this  one  is  for  management  to  establish  a  policy 
that  at  least  one  engineer  in  every  design  group  takes  a  rather 
thorough  reliability  and  maintainability  course  and  becomes  the 
group  expert.  Then  he  in  turn  can  educate  the  group.  Obviously 
this  will  not  be  effective  unless  the  man  chosen  is  one  of  the 
group's  very  beat  people,  who  enjoys  the  respect  of  the  others 
in  the  group.  And  we  said  educate  one  of  the  existing  people, 
who  knows  the  group  technologies,  and  not  to  add  manpower. 

And  lastly,  only  because  there  is  not  much  trouble  with 
contractor  management  recognition  of  the  need  for  it,  there  must 
be  a  core  of  reliability  and  maintainability  specialists,  who 
naturally  must  be  well-educated  in  the  technology.  But  since  it 
is  a  very  youthful  technology,  even  in  its  infancy  in  some  areas 
such  as  structures,  it  is  a  very  fast-growing  technology.  Many 
new  techniques  are  published  each  year,  and  the  number  of 
technical  articles  published  is  fantastic.  The  reliability  and 
maintainability  specialists  must  keep  themsclven  up  to  date  with 
the  technology,  and  serve  ideally  to  in  turn  keep  the  design 
engineers  up  to  date  on  the  techniques  thej  ,se.  And  it  is 

particularly  important  for  them  to  train  new  hives,  as  few 
universities  are  doing  anything  about  it. 
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But  let's  consider  the  design  engineer.  No  matter  how  much  we 
educate  nim  in  the  theorv  and  techniques ,  he  cannot  use  them 
without  the  necessary  tools.  He  cannot  justify  the  time  to 
research  the  literature.  He  cannot  take  i.ime  to  reduce  a 

far -out  mathematically  complex  theory  to  a  simple  equation  that 
fits  his  problem.  He  cannot  use  the  equation  if  he  cannot  lay 
his  hands  on  applicable  and  convincing  data  to  feed  it.  In 
short,  his  primary  tools  are  very  concise  equations,  tables, 
charts,  specs,  organized  references,  etc.,  that  are  convenient 
to  use  every  day.  So  an  educational  program  that  does  not 
concurrently  provide  the  tools  doesn't  buy  anything. 

Figure  21-19  indicates  ^ne  kinds  of  tools  needed,  which  may  be 
in  the  form  Reliability  and  Maintainability  Manuals.  And 
since  the  tools  must  be  concise,  there  must  be  handy  references 
to  permit  him  to  dig  deeper  into  any  specific  aspect  of 
importers:.--  to  his  design.  But  while  we  have  discussed  the  design 
engineers  problem,  there  is  a  corresponding  need  of  the  relia¬ 
bility  and  maintainability  specialists  for  concise  analysis  tools. 
And  there  is  a  need  of  management  for  concise  management  tools. 
Figure  21-20  shows  a  typical  standard  content  of  Manual  sections. 
Many  contractors  have  developed  "Reliability  Manuals"  roughly 
equivalent  to  this  course.  Relatively  few  contractors  have 
developed  manuals  in  a  form  really  useful  to  design  engineers. 

6 ’  TECHNOLOGY  DEVELOPMENT 

In  Section  2.4  we  mentioned  that  although  mhny  of  the  relia¬ 
bility  and  maintainability  techniques  are  well-developed,  others 
are  not.  Structural  design  to  specified  reliability  needs  a 
better  approach.  Data  collection  systems  are  still  inadequate, 
though  we  know  the  data  is  obtainable.  Design  simplification 
techniques,  based  on  reliability  gain,  are  unexploited.  Design 
for  optimum  redundancy  (of  mode,  level,  and  kind)  is  still 
largely  off  the  top  of  the  design  engineers  head,  rather  than 
analytically  logical.  Cost-effectiveness  criteria,  for  day-to- 
day  use  in  hardware  design,  are  undeveloped.  There  are  many 
more  examples. 

Many  small  contracts  have  been  let,  by  various  government 
agencies,  for  the  development  of  handbooks  and  specific 
techniques.  Unfortunately  most  of  these  involve  new  documents 
or  specific  hardware  analyses,  as  opposed  to  actual  design 
achievement  methods.  And  most  of  them  are  actually  initiated 
by  a  contractor,  and  often  the  RFP  is  so  worded  that  very  few 
other  contractors  can  understand  the  scope  or  propose  to  comply. 
There  is  a  need  for  Bureau  development  of  a  reliability  and 
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maintainability  technology  research  program  to  lake  advantage 
of  latent  contractor  specialized  capability. 

But  most  large  contractors  also  conduct  small  reliability  and 
maintainability  technology  development  programs  out  of  their  own 
profits.  A  good  example  (2)  is  the  development  of  a  technique 
for  cost-effectiveness  ranking  of  design  alternatives  to  deter¬ 
mine  optimal  reliability  and  maintainability.  This  of  course 
is  done  (a)  to  provide  an  advantage  in  the  customers'  eyes 
relative  to  the  competition,  (b)  to  provide  more  efficient 
(therefore,  less  costly)  means  of  achieving  required  reliability 
and  maintainability,  and  (c)  to  interest  and  hold  top-notch 
people  between  projects. 

Another  very  important  contribution  to  the  reliability  and 
maintainability  technology  is  contractor  active  participation  in 
government  committee  efforts,  industry-S-government  committees, 
technical  society  conferences  and  symposia,  etc.  Next  to  actual 
application  of  a  new  technique,  there  is  no  better  way  to  its 
development  than  trying  to  sell  it  to  one's  col’eagues  having 
diverse  application  backgrounds  . 

7 •  CHANGE  CONTROL 

Currently  known  by  the  fancier  name  "configuration  management," 
change  control  is  an  old  problem.  It  is  the  control  of  engineer¬ 
ing  design  chanqes  in  such  a  w'ay  that  all  potential  consequences 
are  adequately  considered  before  the  change  is  released,  and  then 
the  careful  documentation  and  assurance  that  every  consequence  in 
design,  procurement,  manufacture,  test,  installation,  operation, 
and  maintenance  is  in  #act  handled  properly. 

Now  change  control  is  not  normally  a  "reliability1  matter,  in 
the  sense  that  an  RSM  group  is  responsible  for  it.  But  all  too 
frequently,  almost  ’normally,"  change  decisions  are  made  without 
adequate  cons iderat ion  of  the  reliability  and  maintainability 
consequences.  And  that  makes  it  an  important  consideration  in 
this  course. 

A  good  contractor  will  enforce  change  control  procedures  that 
prevent  release  of  a  change  until  its  effect  on  (a)  system 
reliability  and  maintainability,  (b)  reliability  and  maintain¬ 
ability  acquisition  cost,  and  (c)  reliability  and  maintainability 
ownership  cost  have*  been  evaluated  and  deemed  acceptable. 
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8.  CORRECTIVE  ACTION  CONTROL 

The  road  to  unreliability  is  paved  with  good  intentions.  Human 
nature  being  what  it  is,  planned  actions  often  get  superseded  or 
forgotten  in  the  subsequent  pressure  of  other  needed  actions. 

Quality  Control  people  have  developed  a  technique,  called 
Corrective  Acti^  Control,  ^hat  very  effectively  prevents  drop¬ 
ping  a  required  action  through  the  crack  in  the  floor.  It  can 
be  and  is  applied  very  effectively  in  enaineering,  especially 
for  design  review  follow-up  as  discussed  in  Chapter  15,  and  failure 
diagnosis  as  discussed  in  Chapter  16. 

Whenever  a  certain  design  or  other  action  is  decided  upon,  to 
achieve  or  preserve  reliability  and  maintainability,  it  is 
given  a  number  and  title  in  a  "Corrective  Action  Log."  The  log 
states  very  briefly  what  action  is  to  be  undertaken,  and  most 
importantly  who  is  respc nsible  for  doing  it.  And  a  single  name 
is  more  effective  than  shared  responsibility.  Then  it  shows  a 
date  by  which  time  it  is  to  be  done. 

The  log  is  published  weekly,  distributed  to  those  whose  names 
are  on  it  for  action.  When  action  does  not  occur  in  a  reasonable 
time,  a  copy  is  distributed  weekly  to  the  delinquent's  supervisor. 
The  result  is  that  people  are  periodically  reminded  of  the  due 
date,  the  action  cannot  be  forgotten,  and  people  like  to  get 
their  names  off  the  list. 

Custody  and  distribution  of  the  Corrective  Action  Log  should  be 
controlled  by  an  independent  group,  so  that  items  cannot  be 
scratched  until  complete.  For  example,  a  decision  to  make  a 
design  change  for  higher  reliability  must  result  in  one  or  more 
sequential  Corrective  Action  Log  items  until  the-  change  is 
comoletelv  implemented .  If  it  is  already  in  production  that 
means  all  retrofits  installed  and  checked  out,  spares  on  hand, 
technical  manuals  changed,  etc. 

9.  SUMMARY 

In  this  chapter  we  have  attempted  to  describe  "typical" 
contractor  organization  to  achieve  the  required  reliability 
and  maintainability.  Work  flow  from  receipt  of  an  PFF  to 
delivery  of  operating  systems  is  shown  in  a  gross  way .  though 
the  details  vary  widely  among  contractors.  In  particular,  the 
reiterative  design  cycle,  alternating  reliability  and  maintain¬ 
ability  achievement  and  evaluation,  is  described. 
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Organization  structure  is  important  to  Br Ships  only  to  assure 
itself  that  it  is  one  of  many  that  are  effective.  But  an 
understand  inn  of  typical  structures  will  also  aid  understanding 
and  ova  1  uat  ion  c  f  pr ::  cos  a  1  s  . 

It  is  shown  that  there  is  usually  good  reason  for  specialized  re¬ 
liability  and  maintainability  groups.  The  Bureau  has  recognized 
this  and  has  started  implementation  in  its  organizations.  The 
need  for  very  clear  policy,  procedures,  and  responsibility  assign¬ 
ment  is  emphasized,  and  typical  allocations  shown. 

Education  is  even  more  effective  o  the  contractor  than  to 
BvShips,  and  must  encompass  management ,  design  engineers,  and 
reliability  and  maintainability  engineers.  And  education  of 
design  engineers  is  worthless  without  the  day-to-day  tools,  or 
concise  manuals. 

Technology  development,  change  control,  and  corrective  action 
control  are  also  discussed,  as  significant,  elements  of  the 
contractor's  activities. 
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Chapter  22 
TASK  DELINEATION 


Although  there  have  been  a  dozen  or  so  government  specifications 
for  reliability  program  management,  DOD  has  consolidated  these 
into  one  concise  codument  (1),  MIL  STD  785,  based  largely  upon 
MIL  R  27542a.  It  may  be  applied  to  all  DOD  military  systems  and 
their  ma  jor  subdivisions.  Ho'-ever  MIL  R  227  32B  (SHIPS )  is  also 
available  (2)  for  electronic  equipment  reliability  and  (3)  MIL 
M  23313A  for  maintainability. 

Like  all  concise  top  documents,  it  cannot  go  into  the  detailed 
requirements  applicable  to  a  specific  system,  thus  is  subject  to 
differences  of  interpretation.  For  any  one  project  or  subsystem 
a  given  785  paragraph  may  or  may  not  make  sense.  And  even  when 
the  paragraph  applies,  the  1epth  cr  level  of  effort  on  it  is 
subject  to  wide  differences  of  opinion.  So  we  must  find  ways  to 
nail  down  exactly  what  is  to  be  done. 

To  the  contractor,  the  language  of  these  specifications  cuts 
across  his  normal  organization  structure.  That  is,  he  must  trans¬ 
late  it  to  tasks  that  he  can  assign  to  specific  groups  with  clean 
lines  of  responsibility.  It  is  in  this  translation  that  undetec¬ 
ted  misunders tanamgs  and  omissions  occur. 

For  vhe  BuShips  engineer,  these  tasks  constitute  (a)  the  basic 
proposed  work  content  to  be  included  in  the  Technical  Development 
Plan  (TOP)  and  (b)  the  work  to  be  specified  to  the  contractor  and 
monitored  thereafter.  Chapter  23  provides  the  TL7  program  lan¬ 
guage  that  integrates  these  tasks,  including  intended  Bureau 
actions  to  assure  successful  completion  of  the  tasks. 

In  tr.is  chapter  we  will  quote  those  sections  of  785  that  result 
in  specific  contractor  tasks,  and  then  suggest  for  each  any  other 
language  that  may  be  used  for  more  specific  requirements.  In  ad¬ 
dition  there  will  be  some  tasks  not  directly  called  out  by  785, 
but  which  experience  shows  are  needed  for  large,  and  some  small, 
projects.  7,11  such  language  applies  to  major  projects,  and  the 
depth  to  ’bich  each  task  is  needed,  if  at  all,  is  discussed  in 
chaptei  23. 

Finally,  most  of  the  tasks  apply  equally  well  to  the  achievement 
of  required  maintainability  or  availability.  Since  as  yet  there 
is  no  DOD  maintainability  standard,  we  have  inserted  the  words 
“(and  maintainability)"  etc.  as  appropriate,  in  parentheses.  It 
is  suggested  that  the  parentheses  be  removed  when  used  for  RFP 
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instructions  for  the  contractors  reliability  and  maintainability 
Program  Plan. 

1 .  PROGRAM  PLAN  IT  DATE 

During  contract  negotiation  many  items  will  be  dropped  and  added. 
After  contractual  authorization  and  during  design  and  production 
there  will  be  changes.  Experience  shows  that  consequent  maintenance 
of  the  program  plan  as  the  current  primary  instrument  of  understand¬ 
ing  between  BuShips  and  the  contractor  is  a  substantial  and  impor¬ 
tant  task. 

"The  contractor  shall  prepare  a  firm  reliability  and  maintain¬ 
ability  Program  Plan  as  a  result  of  contract  negotiation,  and 
update  it  quarterly  tc  agreed-upon  changes.” 


2 .  EDUCATION  and  MANUALS 

A  primary  deterrent  to  reliability  and  maintainability  achieve¬ 
ment  is  simple  management  and  engineering  lack  of  understanding 
of  the  problem  and  what  to  do  about  it.  Very  few,  if  any,  of  the 
contractors  people  have  had  such  training  in  college.  Some  very 
competent  contractors  have  trained  some  of  their  people;  many 
very  competent  contractors  have  not  trained  any.  Thus  the  "in¬ 
doctrination"  (of  management)  and  training  (of  engineers)  must  be 
assured  at  the  very  inception  of  a  program.  See  chapter  21. 
Otherwise  there  will  be  substantial  resistance,  inaction,  redesign, 
excess  cost,  schedule  slippage,  and  poor  reliability  and  maintain¬ 
ability. 

Actually  this  is  part  of  a  bigger  problem  (4)  in  that  "hall  the 
knowledge  of  today's  engineering  graduate  will  be  obsolete  in  a 
decade,  and  half  of  what  he  needs  to  know  then  has  not  yet  been 
discovered."  General  Motors  has  a  full-time  faculty  of  200. 

General  Electric  spends  $45  million  a  year  to  support  courses  for 
35,000  students.  North  American  Aviation  enrollment  is  10,000 
costing  $4.5  million  last  year.  Philosopher-mathematician  Alfred 
North  Vvhitehead  says  "Knowledge  keeps  no  better  than  fish." 

MIL  STD  785  states  (indented  paragraph  numbers  henceforth  are 
those  of  785.  If  in  parentheses  it  is  a  partial  quote); 

”3.5.8  Reliability  Indoctrination  and  Training.  The  relia¬ 
bility  program  shall  contain  provisions  to  supplement  the 
basic  training  and  indoctrination  of  company  and  plant  per¬ 
sonnel  with  reliability  and  maintainability  training  to  assure 
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that  their  skills  and  knowledge  keep  pace  with  advancing 
technology  and  the  requirements  or  peculiarities  of  the 
ay stem  or  equipment." 

Sine®  the  problem  is  often  deeper  than  the  contractor,  these  words 

may  added : 

"The  program  shall  provide  for  corresponding  indoctrination 
and  training  of  appropriate  supplier  personnel." 

It  is  not  enough  to  just  lecture  engineers  on  the  technology,  and 
five  them  references  for  further  reading.  The  average  design 
•nginmor  will  not  begin  to  have  the  time,  nor  the  patience,  to 
research  the  voluminous  literature  to  find  what  he  needs  for  day- 
to-day  design.  It  is  essential  that  they  be  supplied  with  one 
or  more  very  concise  references  manuals,  wherein  they  can  quickly 
find  nearly  everything  they  need  to  know  day-to-day.  Thos  in 
turn  can  reference  other  literature  for  deeper  specialized  anal¬ 
ysis.  So  the  following  may  be  added : 

“The  contractor  shall  provide  one  or  more  concise  reliability 
and  maintainability  reference  manuals,  as  well  as  the  necessary 
data  for  common  general  use  by  all  engineers  on  all  projects. 

If  accessary  he  shall  provide  them  to  suppliars." 

3.  DKSIGM  TO  SPECIFIED  RELIABILITY  AND  MAINTAINABILITY 

Another  major  deterrent  to  reliability  achievement  is  the  general 
lack  of  design  techniques  in  a  form  convenient  to  the  design  en- 
gina«r  for  day-to-day  use.  As  Rear  Admiral  Emerson  Fawkes  has 
stated  {5} : 

"W«  must  obtain  a  major  advance  in  weapon  reliability  and 
maintainability.  It  is  here  that  the  greatest  cost  is 
experienced,  and  here  that  the  greatest  improvement  in  system 
•  f fectiveness  can  be  obtained.  A  five  or  ten  percent 
imawrovwrent  is  not  enough. " 

MIL  STD  7»!S  states: 

(3.3.3)  "  The  contractor  shall  identify  specific  technical 

problems  to  be  solved,  review  problems  considering  program 
requirements,  and  develop  a  detailed  program  to  solve  the 

problems . All  designers  and  associated  personnel  shall 

be  made  aware  of  the  reliability  requirements  pertaining 
to  their  area  of  responsibility  and  shall  be  included  in 
the  information  loop  to  correct  known  deficiencies." 
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"3. 5. 3.1  Where  estimates,  data,  and  experience  indicate  a 
need  for  a  parts  reliability  improvement  program  to  achieve 
desired  system  reliability,  the  contractor  shall  propose  a 
program  to  increase  the  standardization  and  reliability  of 
parts  to  the  required  level." 

"3.5.11  Maintainability.  The  effects  of  the  reliability 
program  on  the  maintainability  of  the  design  shall  be 
considered  during  the  initial  and  subsequent  design  phases 
to  assure  minimum  degradation  to  system  availability." 

"3.5.12  Effects  of  Storage,  Shelf-Life,  Packaging,  Trans¬ 
portation,  Handling,  and  Maintenance.  The  contractor  shall 
determine  by  test  and  analysis,  or  shall  estimate,  the  effects 
of  storage,  shelf-life,  packaging,  transportation,  handling 
and  maintenance  on  the  reliability  of  the  product.  He  shall 
design  the  product  to  withstand  these  effects.  Any  special 
requirements  or  limitations  on  shelf-life,  storage,  packaging, 
transportation,  handling,  and  maintenance  shall  be  made 
known  to  the  procuring  activity." 

The  approaches  that  may  be  used  to  design  to  specified  values  of 
reliability  and  maintainability  are  discussed  in  chapter  13. 
Generally  they  involve  (a)  designing  around  (to  eliminate  or  re¬ 
duce  criticality)  the  questionable  component,  (b)  compensating 
for  its  deficiency  via  redundancy,  etc.,  (c)  improving  it,  (d) 
testing  to  find  out  more  about  it,  etc.  This  is  probably  the  most 
important  single  element  of  the  program  plan.  But  parts  improve¬ 
ment  is  probably  the  most  expensive  and  there f  e  the  last  resorts 

"The  reliability  and  maintainability  program  shall  title  and 
describe  the  anticipated  reliability  and  maintainability 
design  problems,  with  a  statement  of  the  data  and  component 
sources  investigated  and  the  logic  of  such  identification. 

It  will  include  situations  where  either  (a)  the  available 
data  indicates  that  state-of-the-art  components  cannot 
satisfy  the  requirements,  or  (b)  there  is  insufficient  known 
experience  with  the  component  ft*  develop  adequate  confidence 
that  it  will  meet  the  requirements." 

"For  each  reliability  problem  above,  the  program  shall  state 
the  design  approaches  to  be  used,  considering  as  appropriate 
(a)  s implication ,  (b)  standardization,  (c)  parts  selection 

and  application,  (d)  stress/strength  design,  (e)  tolerance 
evaluation,  (f)  failure  rate  prediction,  (g)  human  engineer 
ing,  (h)  failure  cause  &  effect  avoidance,  (i)  preventive 
maintenance,  (j)  producibility ,  (k)  supplier  evaluation  & 
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control,  (1)  evaluation  tests,  (m)  local  environment  control, 
(n)  failure  prediction  devices,  (o)  component  integration, 

(p)  redundancy,  and  (q)  parts  improvement." 

"For  each  maintainability  problem  the  program  shall  state  the 
design  approaches  to  be  used,  considering,  as  appropriate 
(a)  simplication,  (b)  standardized  design,  (c)  modular  design, 
(d)  adjustments,  (e)  failure  effect  provision,  (f)  accessi¬ 
bility,  (g)  safety,  (h)  evaluation  tests,  (i)  identification, 
(j)  total  maintenance  policy,  and  (k)  failure  detection  and 
isolation  devices." 

"For  each  problem  above,  it  shall  state  the  test  or  other 
means  of  verification  to  be  conducted  to  shown  quantitatively 
that  the  design  approach  solves  the  problem,  using  only  short 
one-or-two  sentence  statements  that  reljite  the  problem  to  the 
overall  Verification  Plan  belo^." 


4.  APPORTIONMENT 


Apportionment  of  reliability  and/or  availaoility  (accounting  for 
maintainability)  is  the  "budgeting"  of  the  overall  system  require¬ 
ment  down  to  subsystem  and  component  levels,  so  that  individual 
design  groups  know  what  is  required  of  their  designs.  It  is  done 
very  approximately  at  first,  then  refined  as  the  system  model  and 
prediction  is  developed.  Ultimately  the  prediction  and  apportion¬ 
ment  are  (inversely)  identical.  See  chapter  6.  To  the  extent 
that  design  is  accomplished  within  BuShips ,  so  must  the  apportion¬ 
ment  be  accomplished  by  BuShips.  The  contractor,  however,  will 
nearly  always  have  to  apportion. 

MIL  STD  785  states: 

(3.2.2)  Apportionment  of  reliability  and  maintainability 
requirements  from  the  system  to  system  elements  shall  consider 
complexity  and  importance  (effect  of  failure)  of  the  system 
elements  including  alternative  modes  of  operation." 

"3.5.5  Mathematical  Models.  The  contractor  shall  provide 
mathematical  models  based  on  systems  analysis  to  apportion 
reliability  and  availability  over  major  system  elements; 
and  to  predict  reliability  and  maintainability  at  various 
states  of  design.  The  mathematical  models,  apportionment, 
and  initial  prediction  shall  be  included  in  the  program  plan." 

initial  apportionment,  if  done  right,  can  have  a  tremendous  effect 
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upon  the  design  and  cost  of  the  system  (such  as  choice  of  subsys¬ 
tems  or  components  and  decisions  on  redundancy  needed),  thus 
deserves  some  detailed  attention.  It  also  contributes  to  the 
comparison  of  competitive  contractor  designs  and  competence. 

These  words  may  be  used: 

"The  program  plan  shall  provide  a  block  diagram  of  the  entire 
system  system  at  least  down  to  the  subsystem  level,  and  low'er 
where  feasible,  showing  on  each  block  (a)  its  functional  name 
and  the  apportioned  (b)  reliability,  (c)  criticality  (contri¬ 
bution  to  system  failure  rate) ,  (d)  maintainability,  and/or 

(e)  availability  that  satisfies  the  overall  system  require¬ 
ments.  For  simplicity  he  should  where  feasible  use  (for  b  &  c) 
failure  rate  (per  million  hours),  (for  d)  MTTR  (hours  per 
failure)  and  (for  e)  fraction  of  operating  time,  also  showing 
( fc*  f)  manhours  per  failure.  This  will  facilitate  direct 
addition  and  visualization  of  the  relative  criticality  of 
components.  The  contractor  will  fully  explain  the  methods 
used  for  such  apportionment." 


5.  MODELS  AND  PREDICTION 

As  design  progresses,  a  system  analytical  model  is  developed  and 
refined  to  represent  behavior  of  the  ultimate  system.  When  the 
anticipated  reliability  and  maintainability  of  each  system  func¬ 
tional  block  is  established,  the  model  is  thenceforth  used  to 
periodically  predict  system  re^ability  and/or  availability. 

Such  models  permit  the  design  engineer  to  analytically  test  and 
decide  between  alternatives  before  commitment  to  production,  and 
tell  him  where  his  design  reliability  or  maintainability  is  not 
adequate,  or  may  be  too  good.  They  also  provide  some  management 
visibility  of  progress.  See  chapters  3  and  5,  and  figures  22-8 
and  22-9. 

MIL  STD  785  states: 

"3.2.2  Reliability  Requirement  Studies.  The  reliability  and 
maintainability  program  shall  provide  for  preliminary  and 
continuing  studies  of  reliability  and  maintainability  estimates 
and  achievements.  The  reliability  and  maintainability  program 
for  all  program  phases  shall  provide  for  progressive  refinement 
of  the  reliability  and  maintainability  analysis  and  validation 
of  specified  requirements  for  all  planned  missions  or  opera¬ 
tional  modes  of  the  system.  These  studies  shall  include  de¬ 
finition  of  functional  performance  limits,  duration  of  opera¬ 
tion  in  time  or  cycles,  etc.,  and  the  environmental  conditions 


MODELS  BY  PROGRAM  PHASE 


22-9 


New  design  gui 


22-10 


of  operational  use.  Apportionment  of  reliability  and  main¬ 
tainability  requirements  from  the  system  to  system  elements 
shall  consider  complexity  and  importance  (effect  of  failure) 
of  the  system  elements  including  alternative  mades  of  opera¬ 
tion.  Progressive  reliability  and  maintainability  goals  shall 
be  established  for  each  major  phase  of  a  program  which  are 
phased  with  program  review  points  (3.4)." 

"3.5.2  Furnished  Equipment.  Where  other  equipments,  such  as 
Government- furnished  or  associate  contractor  supplied  equip¬ 
ment  are  to  be  integrated  to  provide  a  complete  operational 
system,  the  contractor  shall  use  known  or  estimated  reliabil¬ 
ity  values  for  these  equipments.  When  such  empirical  data 
are  not  available  through  the  channels  to  which  the  contractor 
has  access,  the  contractor  shall  request  such  data  from  the 
procuring  activity.  The  contractor  shall  report  potential 
reliability  problems  introduced  by  deficient  Government- 
furnished  equipment  or  other  associated  equipment  over  which 
he  has  no  control  and  shall  indicate  and  justify  the  system 
changes  necessary  to  accommodate  or  the  improvement  necessary 
to  make  this  equipment  compatible  with  the  system  requirements." 

"3.5.5  Mathematical  Models.  The  contractor  shall  provide 
mathematical  models  based  on  systems  analysis  to  apportion 
reliability  and  availability  over  major  system  elements;  and 
to  predict  reliability  and  maintainability  at  various  stages 
of  design.  The  mathematical  models,  apportionment,  and 
initial  prediction  shall  be  included  in  the  program  plan." 

Here  again  these  techniques  can  have  a  powerful  effect  on  the 

efficient  and  timely  allocation  of  design  effort,  as  they  can 

show  up  potential  deficiencies  long  before  any  metal  is  bent. 

For  more  detail  these  words  can  be  used: 

"The  analytical  functional  model  of  the  entire  system  shall 
permit  (a)  prediction  of  system  operational  availability  and/ 
or  reliability,  for  each  operational  mode,  as  a  function  of 
its  component  reliabilities  ar.d  maintainabilities,  and  con¬ 
versely  (b)  assessment  of  component  criticality  (increment 
of  system  failure  rate),  and  (c)  prediction  vs.  apportionment 
of  component  reliability  (failure  rates  where  feasible)  and 
maintainability  (MTTR  hours)  and  where  feasible  (d)  manhours 
per  failure.  The  model  shall  include  human  operators  and 
maintenance  personnel  as  components  of  the  system,  taking 
their  reliability  into  account.  The  program  plan  shall  fully 
explain  the  methods  used  to  develop  the  model,  and  the  pro¬ 
cedures  that  will  feed  actual  component  data  into  the  model 
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as  rapidly  as  it  becomes  obtainable,  as  well  as  estimates 
where  necessary." 

'The  model  shall  be  kept  current  as  design  progresses.  The 
contractor  shall  issue  a  monthly  updated  prediction  of  relia¬ 
bility,  availability  and/or  maintainability,  as  appropriate, 
of  the  entire  system,  all  subsystems,  and  all  components. 

He  will  show  current  apportionment  figures  alongside  each 
prediction  figure,  and  highlight  problems." 


6.  COST-EFFECTIVENESS  ANALYSIS 

Cost-Effectiveness  analysis  today  is  rarely  required  and  infre¬ 
quently  used  by  contractors,  --  at  least  by  this  name.  But  there 
is  a  growing  need  for  its  use,  as  discussed  in  chapter  26. 

While  such  analyses  are  necessary  to  (a)  establish  requirements 
(chapter  23),  they  are  also  useful  to  (b)  periodically  re-examine 
requirements  as  design  progress  turns  up  unforeseen  considerations, 
to  (c)  provide  management  visibility  of  progress,  to  (d)  provide 
the  design  engineer  with  a  day-to-day  criterion  for  decisions 
(for  example  if  thereby  he  knew  that  doubling  the  MTBF  of  his 
particular  design  would  save  $100,000  of  total  cost,  he  would  be 
alert  to  such  opportunities  as  he  considers  alternatives),  and 
(e)  provide  a  sound  basis  for  supervisory  allocation  of  design 
manpower  according  to  potential  payoff  to  BuShips  and  the  con¬ 
tractor  . 

Cost-effectiveness  analysis  should  not  be  confused  with  Value 
Engineering  or  Value  Analysis.  While  these  very  important 
techniques  have  the  same  obiectives,  and  contribute  substantially 
^o  cost-effectiveness,  their  almost  universal  application  (0,7) 
is  limited  tc  Acquisition  Cost  reduction  of  paper  and  assembled 
hardware  designs. 

MIL  STD  785  has  no  provision  for  cost-effectiveness  analyses. 

If  it  is  desired,  these  words  may  be  used: 

"The  program  plan  shall  provide  for  continuous  update  of  the 
pre-contract  cost-effectiveness  analysis.  The  contractor 
shall  issue  a  quarterly-updated  (a)  prediction  of  system 
cost-effectiveness,  showing  (b)  the  trend  from  prior  pre¬ 
dictions,  and  (c)  the  total  cost  saving,  if  any,  that  would 
result  from  2-to-l  MTBF  or  MTTR  improvement,  for  each  sub¬ 
system  and  component." 
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FAILURE  MOLL'S  &  EFFECTS  ANALYSIS 


This  is  an  old  qualitative  procedure  that  good  engineers  will  say 
is  done  intuitively  anyway .  What  is  relatively  recent,  besides 
the  name,  is  the  meticulous  and  systematic  detail.  See  chapter 
12.  If  a  reliability  model  is  available,  it  should  be  able  to 
predict  quantitatively  the  effect  of  such  failure  modes.  MIL 
STD  785  para.  3.5.6. 1  (3)  says  only  that  "design  review  analysis 

shall  include,  to  the  v  vtent  applicable,  an  'ysis  of  effects  of 
failure. "  But  it  is  a  widely-used  worth-while  tool.  These  words 
are  recommended: 

"The  program  plan  shall  include  procedures  for  very  systematic 
consideration  of  the  significant  ways  in  >.hich  critical  (whose 
failure  would  cause  system  failure)  components  might  fail,  in 
relation  to  potential  causes  and  effects .  Such  analyses  shall 
be  conducted  during  design,  with  appropriate  design  modifica¬ 
tions  to  minimize  adverse  system  effects,  prior  to  design  re¬ 
view  and  release  to  manufacture." 


6.  STRESS/  STRENGTH  ANALYS  IS 

When  failure  (time)  rate  data  is  not  obtainable,  as  is  often  the 
case  for  mechanical  and  structural  components,  stress/strength 
analysis  may  be  the  only  feasible  method.  however,  it  should  be 
used  wherever  it  is  feasible,  whether  or  not  failure  (4 ime)  rate 
data  is  available.  Traditional  stress,  strength  analysis  using 
"safety  factors"  or  "safety  margins”  is  totally  inn  ’equate  for 
achievement  and  prediction  of  reliability.  Chapter  13  cent  a  ns 
a  design  procedure  and  Chapter  7  the  analysis  techniques.  Since 
MIL  STD  785  has  no  provision  for  -his  moth  xi,  those  words  may  be 
used : 

"For  structures  and  other  designs  here  feasible  the  contrac¬ 
tor  shall  conduct  stress/ strength  analyses  t hat  estimate  re¬ 
liability  from  the  separation  and  variance  of  stress  and 
strength  distributions .  He  will  then  modify  tfv  design  to 
provide  adequate  reliability.” 

9.  HUMAN  FACTORS 


In  quite  a  real  sense  all  failures  are  traceable  to  human  err  r, 
but  unfortunately  not  all  problems  can  be  solved  by  the  human 
factors  ” technology . "  Let's  concentrate  or.  a  few  quite  signifi¬ 
cant  areas.  MIL  STD  78 5  states: 
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"3.5.9  Human  Engineering.  The  reliability  program  shall 
apply  the  principles  of  human  engineering  in  all  operations 
during  design,  development,  manufacture,  test,  maintenance, 
and  operation  of  the  system  or  subsystem.  The  design  shall 
incorporate  human  engineering  features  that  minimize  the 
possibility  of  degrading  reliability  through  human  error. 
Contractor's  human  engineering  personnel  shall  participate 
in  design  activity  and  proposed  tests  to  assure  that,  the 
principles  in  MIL  STL  803  have  been  incorporated  in  design 
and  are  reflected  in  test  plans." 

Since  the  "human  engineering"  phase  no  "tally  does  not  include 
the  human  factors  considerations  in  "  nceptual  design  discussed 
in  chapter  14,  the  following  may  be  added: 

"The  contractor  shall  ar  ilvse  the  functions  required  of  the 
overall  system  and  eacn  subsystem,  to  determine  whether  the 
best  effectiveness  in  relation  to  long-term  cost  (accounting 
for  reliability  and  maintainability)  is  achieved  with  human 
or  hardware  components,  and  modify  the  design  accordingly." 

If  further  translation  in  terms  of  the  specific  contractor  task 
is  desired,  the  following  words  may  be  used: 

"The  contractor  shall  analyse  the  functions  required  of 
human  operators  vs.  skill  level  available,  to  determine 
optimal  display  and  control  cenf igurat ion  and  tne  achievable 
human  reliability  and  maintainability,  and  adjust  the  design 
accord ing  ly . " 

"The  contractor  shall  analyse  the  functions  required  of 
maintenance  personnel  vs.  skill  level,  spares,  and  facilities 
available,  to  determine  optimal  hardware-  configuration, 
diagnostic  aids,  and  achievable  restoration  time,  and  adjust 
the  design  accordingly ." 

1 0 .  DESIGN  REVIEW 

Many  design  engineers  resent  the  idea  that  anyone  else  should  re¬ 
view  and  criticize  their  brain-child.  Yet  it  is  a  fact  that  (a) 
no  engineer  can  know  all  there  is  to  know  about  all  aspects,  'b) 
often  engineers,  under  schedule  pressures,  do  not  have  time  for 
adequate  consideration  of  alternatives  before  proceeding  with 
detail,  (c)  some  engineers  get  so  preoccupied  with  an  "elegant 
approach  that  they  fail  to  see  simpler  approaches,  and  (d)  hardly 
anything  has  ever  been  invented  or  designed  that  an  independent 
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or  "second"  look  cannot  improve  upon. 

The  objective  being  the  best  design  effectiveness  in  relation  to 
long-term  cost,  particularly  as  reliability  and  maintainability 
affect  it,  we  can  hardly  do  less  than  apply  the  best  available 
brains  to  review  every  tentative  design.  After  such  review  the 
design  engineer  (or  his  supervisor)  can  decide  exactly  what  re¬ 
commendations  to  adopt,  but  not  before.  See  chapter  15. 

A  few  military  specifications  require  that  a  military  representa¬ 
tive  be  invited  to  every  review.  There  have  been  a  number  of  oc¬ 
casions  where  the  adoption  of  a  recommended  design  improvement 
resulted  in  the  military  officially  asking  for  some  money  back  on 
the  ground  that  the  original  design  effort  was  improperly  executed. 
Needless  to  say,  this  has  the  effect  of  inhibiting  future  improve¬ 
ment  recommendations,  or  complete  nullification  of  the  benefit  of 
design  review.  Perhaps  some  contractual  protection  of  the  contrac¬ 
tor  will  solve  the  problem,  as  military  participation  is  other¬ 
wise  very  beneficial.  MIL  STD  785  states: 

"3. 5. 6.1  Periodic  design  reviews  for  reliability  (and 
maintainability)  and  evaluation  of  designs  shall  be  conducted 
as  an  integral  part  of  the  contractor's  engineering  design 
review  and  evaluation  pro  cdures .  These  reviews  shall 
evaluate  the  achievemen  f  reliability  (and  maintainability) 
relative  to  the  reliability  (and  maintainability)  goals 
established  for  each  major  phase  and  review  point  of  the 
contract;  with  contractor  evaluation  before  designs  are 
finalized.  The  reliability  and  maintainability  design  review 
analyses  shall  include,  to  the  extent  applicable: 

Not e  that  design  review  for  reliability  and/or  maintainability 
separate  from  other  design  review  is  not  desirable,  as  it  would 
introduce  unnecessary  cost  and  delay.  MIL  STD  785  paragraph 
3. 5. 6.1  goes  on  t  say: 

(1)  Reliability  (and  maintainability)  estimates  based  upon 
prediction  (785  suggests  MIL  STD  756  and  MIL  HDBK  217,  but 
NAVSHIPS  93820  applies  to  BuShips)  and  accumulated  test  data. 
Estimates  shall  be  made  for  each  mode  of  operation. 

(2)  Review  of  potential  design  or  production  problem  areas. 

(3)  Analysis  of  effects  of  failure. 

(4)  Identification  of  the  principal,  critical  items  in¬ 
hibiting  reliability  (and  maintainability)  achievement. 
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(5)  The  effects  of  engineering  decisions  ana  tradeoffs  upon 
reliability  (and  maintainability)  achievements,  potential  and 
growth . 

The  program  plan  shall  specify  appropriate  personnel  from  the 
contractor's  reliability  (and  maintainability)  organizations 
who  shall  participate  in  the  design  reviews  and  denote  approval 
by  signature.  These  reviews  shall  be  continuing  in  nature  to 
provide  for  the  earliest  possible  detection  and  correction  of 
any  potential  deficiencies.  A  system  shall  be  established 
and  maintained  by  the  contractor  to  assure  reliability  (and 
maintainability)  participation  4 n  control  of  designs,  speci¬ 
fications,  drawings,  and  all  changes  thereto. 

The  design  review  shall  compare  the  design  with  previously 
defined  qualitative  and  quantitative  requirements.  The 
results  of  the  review  shall  be  documented. 

The  Procuring  activity  shall  be  notified  at  least  10  days  prior 
to  each  scheduled  formal  design  review  (as  distinguished  from 
continuing) ,  to  permit  procuring  activity  participation.  The 
minutes  of  such  revi.ev.s  shall  be  made  available  to  the  pro¬ 
curing  activity  upon  request." 

Experience  has  shown  that  design  reviews  literally  complying  with 
the  above  may  ..till  be  quite  ineffective.  There  must  be  a  clear 
policy  on  just  what  designs  will  be  reviewed,  to  what  depth ,  when , 
and  by  whom.  Thus  for  more  specific  instruction  in  terms  of  the 
contractors  task,  these  words  have  been  found  effective: 

"The  contractor  shall  conduct  design  reviews  of  all  new  designs 
and  design  changes,  as  well  as  any  other  designs  whose  relia¬ 
bility  or  maintainability  are  either  unknown  or  suspect ,  in¬ 
cluding  review  of  ail  interfaces  and  the  c  ffect  of  env: ronment 
surrounding  each  subsystem,  component,  and  port." 

"He  shall  conduct  suck  reviews  at  least  of  (a)  complete  con¬ 
ceptual  design  of  systems  and  subsystems,  prior  to  hardware 
component  selection  and  detailed  design,  and  of  (b)  complete 
hardware  design  prior  to  release  for  procurement  and  manu¬ 
facture.  However  they  should  be  planned  for  whatever  design 
level  or  grouping  of  components  will  most  economically  pro¬ 
vide  adequate  review  of  each  new  design  and  new  interfaces 
only  once. 

"All  design  reviews  shall  be  scheduled  to  follow  appropriate 
design  completions,  allowing  scheduled  time  and  budget  for 
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design  changes  resulting  from  the  review.  Review  partici¬ 
pants  time  must  be  budgeted,  but  the  cost  is  normally  more 
than  offset  by  retrofit  cost  savings.  The  cost  can  be  min¬ 
imized  through  use  of  standard  checklists  during  design,  which 
are  then  reviewed  by  the  specialists  prior  to  and  during 
review. " 

"Every  design  review  shall  specifically  consider  quantitative 
reliability  and  maintainability  relative  to  established  re¬ 
quirements,  as  well  as  acquisition  and  ownership  costs.  Any 
deficiency  from  requirements ,  as  well  as  all  recommended 
design  changes,  shall  be  carried  as  Corrective  Action  Contiol 
items  (see  13.0)  until  the  deficiency  or  recommendation  is 
resolved . ,! 

"Design  review  participation  shall  be  limited  to  small  groups 
including  the  responsible  design  engineer  and  appropriate 
specialists  from  other  than  the  responsible  design  group, 
specifically  including  the  chairman  and  at  least  one  relia¬ 
bility/maintainability  specialist. " 


11.  PARIS  CONTROL 

Many  contractors  have  excellent  parts  control  procedures,  but 
many  do  not.  The  basic  problem  is  to  prevent  the  selection  of 
parts  about  which  there  is  inadequate  knowledge.  It  is  particu¬ 
larly  serious  in  the  electronics  or  mechanisms  areas,  where  the 
green  young  engineer  can  become  sold  on  an  elegant  new  part  in  the 
vendors  catalog.  It  is  “just  what  he  needs"  but  has  no  history, 
no  pedigree,  no  MTBF  rating.  See  chapter  18.  MIL  STD  785  states: 

"3.5.3  Parts  Reliability.  Parts  shall  not  oe  used  without 
knowledge  of  their  capabilities  and  reliability  potential 
determined  from  current  or  previous  testing.  Information 
shall  be  sought  or  generated  on  stress  levels  and  limits  of 
application  as  well  as  on  failure  rate.  Available  data  and 
central  information  facilities  shall  be  utilized  to  avoid 
needless  duplication  of  testing.  In  using  existing  data, 
the  risk  and  limitations  of  extrapolating  part  performance 
data  at  one  set  of  environments  to  that  expected  at  a  different 
set  of  environments  shall  be  recognized  and  documented.  The 
best  available  estimate  or  determination  of  failure  rate  for 
each  part  type  shall  be  made;  the  part  vendor's  accumulated 
test  history  under  part  specifications  requiring  failure  rate 
verification  shall  be  sought.  Reported  measure  of  achieved 
reliability  should  not  be  based  upon  short  duration  tests 
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which  predominately  measure  performance.  If  time  does  not 
permit  adequate  testing  at  advanced  ayes,  the  contractor  shall 
show  the  age  range  actually  tested  and  shall  justify  use  of 
such  data." 

(3. 5. 3.1)  "A  preferred  parts  list  shall  be  maintained  and 
utilized  as  a  source  of  high  reliability  parts." 

In  terms  of  the  contractors  tasks  the  following  detail  may  be 

added  if  necessary: 

"The  program  plan  shall  provide  for  a  Parts  Control  activity 
responsible  for  (a)  selection  and  cataloging  of  preferred 
parts  that  must  be  used  by  design  engineers  in  preference  to 
all  other  parts  wherever  feasible,  (b)  approval,  or  securing 
BuShips  approval,  of  the  use  of  each  non-standard  part  in 
each  application,  (c)  review  of  all  parts  application  in 
Design  Review,  (d)  consultation  to  design  engineers  on  parts 
reliability  and  maintainability,  availability  and  selection, 

(e)  collection  and  dissemination  of  parts  data  needed  by 
design  engineers,  including  failure  rate  and  cost,  (f)  writing 
all  parts  specifications  to  vendors,  using  military  standard 
format  where  feasible,  (g)  controlling  engineering  stockroom 
content  to  minimize  non-standard  parts,  (h)  providing  parts 
handling  policy  and  procedure,  and  (i)  providing  traceability 
policy  and  procedure,  so  that  failed  parts  may  be  traced  by 
serial  or  lot  number  to  eliminate  their  doubtful  brothers." 


12. 


REPORTS  &  PROJECT  REVIEW 


Most  contracts  generate  volumes  of  reports  that  never  get  read, 
or  at  least  by  the  people  who  can  understand  the  significance  and 
take  action.  The  information  they  contain  may  be  important,  but 
is  not  in  a  form  that  a  busy  man  can  take  time  to  read,  and  the 
distribution  (because  of  report  size  and  cost)  may  be  inadequate. 
And  while  BuShips  needs  regular  reports  from  the  contractor  to 
control  the  program,  the  contractors  internal  distribution  of 
significant  reports  has  far  more  impact  on  efficient  and  timely 
action.  MIL  STD  785  states: 

"3.4  Program  Review.  The  reliability  (and  maintainability) 
program  shall  be  organized  and  scheduled  to  permit  the  con¬ 
tractor  and  the  procuring  activity  to  review  its  status,  in¬ 
cluding  results  achieved,  at  preplanned  steps  or  chef iooints. 
This  formal  review  and  assessment  of  reliability  (and  maintain- 
bility)  normally  will  be  conducted  at  major  program  points 
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and  these  points  will  be  established  by  the  procuring  activity 
during  negotiations.  As  the  program  develops,  reliability 
(and  maintainability)  progress  shall  be  assessed  by  use  of 
such  information  as  predictions  of  reliability  (and  maintain¬ 
ability)  and  results  of  reliability  (and  maintainability) 
design  reviews  and  tests  including  effects  of  human  perform¬ 
ance. 

More  specifically,  the  specification  may  state  the  specific  in¬ 
formation  needed  for  the  OPNAV  3910.15  reports,  as  well  as  for 
contractor  internal  control: 

"The  contractor  shall  conduct  a  summary  reporting  system 
which  provides  to  each  engineering  supervisor  a  monthly  one- 
page  updated  prediction  of  the  availability  [/0)  ,  reliability 
(failure  rate)  and/or  maintainability  (MTTR)  of  each  design 
for  which  he  is  responsible,  with  'apportioned  values  shown 
beside  each  figure,  flagging  adverse  discrepancies.  It  shell 
show  the  prior  month  and  quarterly  values  to  indicate  trend. 

The  contractor  shall  combine  the  abcve  reports  for  higher- 
level  one-page  monthly  reports  of  subsystem  and  overall  system 
availability,  reliability,  and/or  maintainability,  for  con¬ 
tractor  management  and  for  regular  reporting  to  BuShips." 

"The  contractor  shall  provide  a  monthly  report  of  audited 
task  progress  via  (a)  a  status -vs .-plan  copy  of  the  task  Sched¬ 
ule  shown  in  Chapter  23,  and  (b)  a  report  on  each  Task 

of  the  Program  Plan,  of  not  >  ore  than  one  page  per  task,  for 
BuShips  and  contractor  management,  with  copies  of  each  page 
to  contractor  supervision  who  should  take  any  needed  correc¬ 
tive  action." 

If  the  contractor  plans  to  conduct  a  running  cost-effectiveness 
evaluation.  Figure  22-19  shows  a  compact  type  of  such  report. 

"If  available,  the  contractor  shall  similarly  show  the 
cost-effectiveness  trend  quarterly." 


13.  CORRECTIVE  ACTION  CONTROL 

Human  nature  being  what  it  is,  many  recognized  problems  never  get 
fixed  because  they  are  forgotten  in  the  pressure  of  bigger  pro¬ 
blems.  A  very  effective  solution  to  this  problem  is  a  fairly 
automatic  problem  logging  system,  with  a  named  individual  respon¬ 
sible  to  fix  each  problem.  When  such  "needle"  logs  are  regularly 
issued,  audit  of  progress  is  simple,  and  most  ^eople  want  to  get 
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off  the  hook.  MIL  STD  785  states: 

(3.3.3)  ’’Records  shall  be  maintained  on  the  status  of  actions 
to  resolve  problems." 

Since  the  above  words  require  no  more  than  paperwork  recording 
what  happened,  as  opposed  to  methods  for  assuring  action,  the 
Project  Engineer  may  wish  to  pin  it  down  further: 

"The  contractor  shall  conduct  a  Corrective  Action  Control 
System  wherein  (a)  every  predicted  failure  to  achieve  required 
reliability  and  maintainability,  (b)  every  unresolved  design 
review  recommendation,  and  (c)  every  production  and  operational 
hardware  failure  is  logged  and  remains  as  an  'action  required' 
item  until  resolved  to  the  satisfaction  of  the  reliability  and 
maintainability  group(s).  Each  item  will  show  the  name  of 
one  individual  directly  responsible  for  resolution,  and  the 
date  by  which  resolution  is  planned.  In  the  case  of  produc¬ 
tion  hardware  failures,  resolution  must  include  steps  to  pre¬ 
vent  the  same  failure  again  in  that  or  other  hardware." 


14.  CHANGE  AND  CONFIGURATION  CONTROL 

The  problem  of  keeping  track  of  detailed  changes,  their  inter¬ 
faces,  and  their  impact  on  logistics  for  a  complex  system  is  a 
big  task.  It  goes  far  beyond  reliability  and  maintainability  con¬ 
siderations,  yet  has  quite  an  impact  thereon.  See  chapter  21. 
Unless  control  procedures  are  established,  change  control  groups 
often  make  decisions  without  realizing  the  consequences: 

"The  contractor  shall  provide  control  procedures  whereby 
design  and  configuration  changes  cannot  be  approved  without 
quantitative  consideration  of  resultant  reliability  and 
maintainability  of  the  design  and  its  interfaces,  in  the 
local  environment.  The  procedures  must  insure  that  such 
changes  are  immediately  incorporated  into  the  prediction 
model. " 


15.  SUPPLIER  CONTROL 

Since  subcontractors  and  vendors  normally  provide  the  bulk  of  the 
hardware,  and  often  substantial  portions  of  the  design,  it  is  im¬ 
perative  that  all  provisions  of  the  Program  Plan  be  executed  by 
all  suppliers  to  an  appropriate  depth.  See  chapter  19.  This 
appropriate  depth  is  sometimes  difficult  to  determine,  and  the 
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contractor  can  only  ask  himself,  "If  I  were  designing  and  manu¬ 
facturing  this  component,  knowing  what  I  do  about  my  application 
of  it,  what  tasks  would  I  undertake  to  make  very  sure  of  getting 
the  required  reliability  and  maintainability?" 

There  is  also  the  problem  of  the  excellent  supplier  who  says  in 
effect,  "I  know  my  product  is  good,  and  I  don't  need  your  business 
so  much  that  I  want  to  bother  with  this  reliability  legerdemain." 

If  he  is  the  only  source,  it's  a  tough  nut.  But  some  contractors 
are  pretty  resourceful.  MIL  STD  785  states: 

"3.5.7  Supplier  and  Subcontractor  Reliability  (and  Maintain¬ 
ability)  Programs.  The  contractor  shall  be  responsible  for 
assuring  that  supplier’s  and  subcontractors'  achieved  relia¬ 
bility  (and  maintainability;  levels  are  consistent  with  over¬ 
all  system  requirements.  The  contractor  shall  impose,  direc¬ 
tly  or  indirectly,  quantitative  reliability  (and  maintainabil¬ 
ity)  requirements  and  acceptance  criteria  on  all  echelons  of 
supp'iers  and  subcontractors;  and  shall  incorporate  applicable 
portions  of  this  standard  in  subcontracts  and  purchase  orders. 
The  reliability  (and  maintainability)  program  of  the  contrac¬ 
tor  shall  contain  provisions  for  surveillance  of  supplier  and 
sub  ontractor  reliability  (and  maintainability)  activities 
inducing  failure  reporting.  The  surveillance  shall  consist 
of  but  not  limited  to  such  items  as  maintaining  a  supplier 
selection  program  based  upon  review  of  the  supplier’s  relia¬ 
bility  (and  maintainability)  program,  quality  control  system, 
examination  of  his  facilities,  and  past  performance,  to  assure 
that  suppliers  are  capable  of  attaining  and  maintaining  the 
required  level  of  reliability  (and  maintainability).  The  con¬ 
tractor  shall  take  all  actions  necessary  to  assure  that  no 
changes  made  by  any  supplier  will  reduce  reliability  (or 
maintainability)  of  the  system.  Records  of  each  supplier's 
performance  shall  be  maintained  and  reviewed  with  him  period¬ 
ically.  " 

In  terms  of  the  contractors  task,  a  comprehensive  program  would 
be  established  with  these  words: 

t 

"The  contractor  shall  conduct  a  survey  of  the  reliability 
and  maintainability  capability  of  each  finally-considered 
supplier  to  evaluate  (a)  his  design  engineering  knowledge 
of  reliability  and  maintainability  requirements,  (b)  the 
means  to  achieve  such  requirements,  (c)  t-he  analysis  techniques, 
(d)  his  management  understanding  and  support  of  the  tasks  re¬ 
quired,  and  (e)  existing  conduct  of  such  tasks.  The  survey 
sha  _  consider  the  above  reliability  and  maintainability  cap- 
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ability,  as  well  as  quantitative  evidence  of  achieved  MTBF 
and  MTTR,  as  prime  factors  in  the  selection  of  every  supplier." 

"For  every  suppliers  component  whose  criticality  (effect  on 
system  reliability)  exceeds  a  predetermined  level,  the  con¬ 
tractor  shall  i...:1  ’de  in  the  specification  to  the  supplier 
the  required  quantitative  reliability  and  maintainability, 
and  the  test  or  other  criteria  by  which  they  will  oe  verified 
prior  to  shipment.  Specifications  detailing  all  technical 
requirements,  including  tolerances,  shall  be  used  for  ail 
critical  components  whose  failure  would  cause  system  failure- 
ordering  by  vendor  catalog  number  alone  is  prohibited." 

"The  contractor  shall  require  reliability  and  maintainability 
test  data,  and  monthly  predictions  of  product  reliability 
and  maintainability  from  each  supplier  of  a  component  whose 
applied  criticality  exceeds  the  predetermined  level,  whether 
in  design  or  production.  He  shall  require  monthly  progress 
reports  of  each  such  suppliers  Program  Plan  task,  and  take 
appropriate  action." 

"The  contractor  shall  conduct  .e-  ,urveys  every  6  to  12  months, 
identical  to  the  pre-award  sur ;e  and  evaluation.  He  will 
maintain  for  each  supplier  a  currer  -  rating  of  reliability 
and  maintainability  (a)  engineerin<  capability,  (b)  task 
performance,  and  (c)  actual  quantitative  achievement  vs. 
prediction,  for  use  in  future  source  selections." 

16.  MANUFACTURING  R  &  M  CONTROL 

Once  a  design  is  released  to  manufacture,  it  has  an  inherent  re¬ 
liability  that  will  either  be  achieved  through  good  Quality  con¬ 
trol,  or  degraded  through  lack  of  it.  See  chapter  20.  While: 
Quality  Control  is  outside  the  scope  of  this  course,  there  are  a 
few  tasks  that  otherwise  might  not  be  part  of  the  QC  program. 

Here  MIL  STD  785  refers  to  "Critical  Items"  by  which  i.«  meant 
"items  whose  failure  would  cause  system  failure."  In  many 
systems  this  can  be  80%  of  the  hardware,  so  that  the  "special 
handling"  aspect  becomes  meaningless,  and  such  requirements  are 
thus  sometimes  waived.  MIL  STD  785  states: 

"3.5.4  Critical  Items.  The  contractor  shal1  establish  an 
effective  method  for  identification,  control  and  special 
handling  of  critical  p^rts,  components,  subsystems  or  other 
end  items  from  design  through  final  acceptance.  Such  methods 
shall  be  described  in  the  contractor's  formal  policies  and 
procedures  to  assure  awareness  by  all  affected  personnel 
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{e.g.  design,  purchasing,  manufacturing,  inspection,  test, 
handling,  etc.)  of  the  essential  and  critical  nature  of  such 
items.  The  methodology  used  in  generating  the  critical  it«a 
list  shall  be  furnished  to  the  procuring  activity.  The 
method  used  and  the  list  subsequently  generated  shall  be 
subject  to  review  and  evaluation  of  the  procuring  activity." 

"3.5.14  Manufacturing  Controls  and  Monitoring.  The  contractor 
shall  have  a  planned,  controlled  and  scheduled  system  of  pro¬ 
duction  control  and  monitoring  to  assure  that  reliability 
(and  maintainability)  achieved  in  design  is  maintained  during 
production. " 

When  reliability  models  are  used,  every  component  can  be  assigned 
a  "criticality"  number,  the  simplest  form  of  which  is  merely  th« 
system  failure  rate  increment  due  to  all  of  that  particular  com¬ 
ponent,  considering  its  own  failure  rate  and  the  way  it  is  used. 
Thus  the  above  "critical  item"  control  is  better  handled  as  con¬ 
trol  of  items  having  a  criticality  above  a  predetermined  value. 
Other  aspects,  in  terms  of  the  contractors  detailed  tasks,  may  be 
worded : 

“The  contractor  shall  plan,  control,  and  audit  the  procure¬ 
ment,  manufacturing,  test  and  transportation  program  so  that 
(a)  only  known-reliability  and  traceable  parts  and  component* 
can  be  used  (b)  parts  and  components  are  packaged  and 
handled  in  a  manner  that  does  not  degrade  reliability  by 
shock,  wear,  etc.,  (c)  each  component  carries  with  it  a  log 
of  the  serial  or  lot  numbers  of  its  constituents,  and  (d)  a 
log  of  every  test  showing  the  operational  time  duration,  stress, 
environment,  failure  time  and  descriptions,  and  the  downtime 
actions  taken,  with  time  for  each,  (e)  no  overstress  is 
applied  to  deliverable  items,  and  (f)  storage,  shelf-life, 
and  transportation  do  not  degrade  reliability." 


17.  FAILURE  DIAGNOSIS 

Failure  in  design  evaluation,  manufacture,  test,  or  operational 
use  may  or  may  not  be  due  to  faulty  parts,  since  a  technician  may 
have  adjusted  something  improperly,  or  let  his  screwdriver  fall 
into  a  sensitive  circuit.  And  usually  he  won't  admit  it.  Also 
operators  have  been  known  to  make  mistakes,  reporting  a  failure 
to  protect  their  hides.  But  without  human  error,  and  without 
faulty  parts,  components  can  still  fail  to  work  due  to  design 
tolerances  o-  interfaces  between  parts.  See  chapter  16. 


But  if  reliability  is  important,  there  must  bo  one  ironclad  rule 
about  all  failures.  Never,  but  never,  should  a  failure  go  un¬ 
diagnosed  ,  nor  unrecorded.  F.  <  the  failure,  yes,  but  it  is  far 
more  important  to  take  meticulous  steps  to  prevent  its  recurrence 

MIL  STD  785  states: 

"3. 5. 3. 2  Emergency  Reporting  of  Defective  Parts.  When  a  MIL 
specification  or  a  MIL  part  is  deemed  suspect  by  the  develop¬ 
ment  contractor,  the  contractor  shall:  (a)  indicate  reason 
with  supporting  evidence  of  this  conclusion,  (b)  perform 
failed  part  diagnosis  and  analysis  of  those  parts  deemed 
suspect  development,  acceptance  tests,  and  other  rt  'ated 
activities,  (c)  whenever  possible,  reach  a  conclusion  relative 
to  the  cause  of  failure,  and  (d)  report  by  most  expeditious 
means  to  the  procuring  activity  with  concise  support inc  data 
when,  and  only  when,  it  has  been  concluded  that  a  part  is  un¬ 
satisfactory  for  any  of  the  following  reasons: 

(1)  A  part  which  was  accepted  as  meeting  a  MIL  specifi¬ 

cation  but  which  failed  to  perform  to  expectations ,  such 
failure  <-o  be  (attributable  to:  (a)  manufactur¬ 

ing  procedures,  choice  of  material,  or  design  of  part,  or 
(b)  test  and  inspection  disciplines. 

(2)  A  military  part  specification  which  is  inadequate  in 
that  it  (a)  dees  not  take  advantage  of  the  state-of-the- 
art,  (b)  requires  amendment  to  encourage  advancement,  of 
the  state-of-the-art,  or  (c)  requires  revision  for  clarity. 

"3.5.15  Failure  Data  Collection,  Analysis,  and  Corrective 
Action.  (a)  The  contractor  shall  have  and  shall  require 
major  subcontractors  to  have  a  closed-loop  system  for  collec¬ 
ting,  analyzing,  and  recording  all  failures  that  occur  luring 
phases  of  tests  required  for  system  elements  including  those 
that  are  performed  in-plant  and  at  installation  sites.  The 
contractor  shall  describe  his  failure  reporting  procedures, 
including  flow  charts,  for  the  analysis,  feedback  and 
corrective  action  as  part  of  the  program  plan  (see  3 . 3 .  .>  i  . 

The  failure  reporting  system  shall  include  provisions  to 
assure  that  effective  corrective  actions  are  taken  vn  a 
timely  basis  to  reduce  or  prevent  repetition  of  the  failures. 
The  contractor  shall  establish  scheduled  audits  to  review  all 
open  reports,  analyses,  dates  for  corrective  action  and 
report  all  delinquencies  to  management." 

"(b)  The  contractor  shall  commence  failure  reporting  with 
initial  development  testing  or  operation  including  operating 
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equipment  at  receiving  inspection,  at  a  vendor's  plant  in 
final  assembly  checkout ,  or  during  acceptance  testing.  An 
unscheduled  ad iustment ,  other  than  a  calibration  made  during 
other  maintenance  because  of  convenience,  shall  be  defined  as 
a  failure  for  reporting  purposes.  Failure  of  components  pi ior 
to  incorporation  into  an  assembly  shall  be  recorded  separately 
and  reported." 

"(d  The  contractor  shall  submit  failure  report  summaries 
as  specified  by  the  procuring  activity." 

18-  DATA  ACQUISITION  c.  REDUCTION 


Without  convincing  evidence  that  his  beloved  brain-child  is  un¬ 
reliable,  the  design  engineer  cannot  take  seriously  the  claim  that 
higher  reliability  is  needed.  That  evidence  is  operational  stress 
time  and  failure  data.  The  source  of  such  data  can  only  be  the 
user  of  the  system,  in  this  case  the  Navy.  A  real  solution  to 
the  problem  is  not  yet  in  sight,  though  several  groups  are  working 
hard  on  it.  See  Chapter  9. 

One  problem  is  that  the  maintenance  technician  himself  is  never 
a  dependable  data  collector.  He  often  has  tremendous  pressure 
to  restore  operation,  ignoring  data  needs  until  a  time  that  it 
can  no  longer  be  obtained.  One  survey  (8  p.2 17  No.  10)  states 
that  only  10  to  20  -  of  N  ivy  failures  are  getting  reported.  See 
Chapter  26  ,  Sect  i oris  8 . 1 0-8 . 12  . 

Another  problem  is  that  overlapping  needs  for  data  will  result  in 
excessive  maintenance  site  paperwork  unless  they  are  consolidated 
into  a  simple  format.  Another  is  that  even  when  data  gets  back, 
to  a  contract  r ,  often  he  is  not  funded  to  analyze  it  properly 
it  properly  for  BuShips  or  even  his  own  future  guidance.  Mil- 
Std-"!85  stales: 

(3.5.151  "The  contractor  shall  explain  the  m.ethoo  by  which 
failure  repmits  ,ai  >•»  init  iated.  Analysis  and  recording  of 
failures  shall  di f fert nt iate  between,  but  not  be  restricted 
to,  those  due  t  equipment  failure  and  those  due  tc-  human 
error  in  designing,  process ing ,  handling,  transporting, 
storing,  maintaining,  and  operating  the  equipment.  Flapae-d 
time  indicators  or  event  counters  shall  be  utilized  or  a 
log  shall  be  maintained  to  report  accumulated  operation 
time  or  operation  cycles  on  system  components  that  are  time 
or  operation-cycle  sensitive.  The  failure  reporting  system 
shall  be  designed  to  be  compatible  with  the  maintenance 
data  collection  system  of  the  procuring  or  using  activity  so 
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that,  as  the  system  nears  the  operational  inventory  phase, 
transition  to  in-service  failure  reporting  can  be  accomplished 
with  the  minimum  disturbance  and  maximum  continuity  of  effort." 

Long  before  the  contractor  has  any  hardware  to  fail,  his  design 
engineers  need  the  best  obtainable  data  in  order  to  achieve  the 
required  reliability  and  maintainability.  Then  as  the  hardware 
is  produced  he  needs  it  to  evaluate  test  results,  and  then 
operational  data  permits  refinement  of  that  and  future  designs. 
BuShips  can  get  better  designs  if  it  is  in  a  position  to  offer: 

"BuShips  will  provide  pertinent  obtainable  operational 
reliability  (stress  time  and  failures),  maintainability 
(MTTR ) ,  manhours/failure  and  cost  ($  per  failure  other 
than  manhours)  aata  to  the  contractor.  During  design, 
manufacture,  test  and  checkout,  it  will  be  on  components 
similar  to  those  to  be  furnished  to 'the  contractor.  During 
operational  use  it  will  be  on  the  system,  subsystems,  and 
components  furnished  by  the  contractor,  furnished  period¬ 
ically  until  the  system  is  retired." 

Another  problem  is  that  only  a  few  contractors  have  developed 
continuously-operating  data-reduction  systems  that  provide 
information  to  design  engineers  in  convenient  form.  See  chapter 
18  section  4.1  for  such  a  system  now  under  development . 

"The  contractor  shall  collect  corresponding  data  from  his  own 
design,  suppliers,  production,  test,  and  checkout  activity. 

Ke  shall  combine  this  with  data  furnished  by  BuShips  to  main¬ 
tain  current  records  of  reliability,  maintainability  and  total 
cost  of  all  components,  with  due  regard  to  environments  in¬ 
volved.  This  shall  be  made  available  for  convenient  use  by 
all  design  engineers  on  current  and  subsequent  system  designs.' 

"He  shall  analyze  such  data  and,  using  his  reliability  and/or 
availability  model  of  the  system,  quarterly  report  the  current 
■system  and  subsystem  reliability  and  availability  and  (if 
available'  cost-effectiveness  to  the  Bureau." 

19 .  VERTFICATI TN 

Seldom  can  we  wait  until  an  entire  system  is  assembled  to  find 
out  by  test  whether  it  has  adequate  reliability  and  maintain¬ 
ability.  For  by  then  the  accumulation  of  errors  would  almost 
certainly  guarantee  serious  schedule  slippage  and  cost  overrun. 

On  the  other  hand  we  cannot  afford  a  separate  sequential  test 
for  every  unknown  or  unverified  parameter.  Thus  most  contractors 
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develop  "integrated  test"  plans  which  seek  to  get  the  .iiaximum 
information  but  of.  the  fewest  possible  number  of  tests,  and  at  the 
earliest  feasible  time  to  permit  scheduled  correction  of  defi¬ 
ciencies.  Mil-Std-78"  states: 

"3.5.1  Test  Requirements  for  Development  Qualification  and 
Acceptance.  A  Planned  and  scheduled  program  of  functional 
and  environmental  testing  of  equipment  shall  be  conducted 
during  design  and  development  phases  to  estimate  achieved 
reliability  and  to  provide  feedback  of  data  as  a  basis  for 
making  reliability  improvements .  The  development  testing 
program  shall  confirm  adequacy  of  selection  of  components 
and  parts,  determine  capabilities  and  safety  margins, 
evaluate  drifts  of  component  parameters  with  time,  and 
determine  failxire-modes  and  relative  failure-rates." 

"If  such  data  are  not  available,  all  items  of  the  system 
determined  by  the  reliability  studies  (3.3.2  and  3.3.3)  to 
have  a  significant  bearing  on  inherent  reliability  shall  be 
tested  early  in  the  development  program,  unless  other  valid 
proof  of  adequacy  can  be  presented." 

Keeping  in  mind  the  discussion  of  "critical  item"  vs.  "critical¬ 
ity"  in  Section  16.0  above,  Mil-Std.-785  states: 

"3. 5. 1.1  Environmental  Requirements  for  Equipment  Design 
and  Testing.  If  maximum  environmental  stress  conditions  have 
not  been  established  by  the  producing  activity  these  shall  be 
estimated  from  experience  on  past  programs,  and  a  test  program 
for  development,  qualification,  and  acceptance  shall  be 
generated  on  this  basis.  Development  and  qualification  tests 
shall  be  planned  to  evaluate  the  adequacy  of  design  of  equip¬ 
ment  for  the  expected  conditions  in  the  use-envircnment  (e.g., 
ground  operation,  launch,  flight  and  orbit).  The  test  plans 
shall  include  consideration  of  equipment,  location,  insulation 
shock-mounting,  truss  mounting,  etc." 

"Environmental  problem  areas  shall  be  identified  at  the 
system,  subsystem,  component  and  part  level,  and  the  effects 
of  these  problems  on  system  reliability  shall  be  estimated 
on  equipments,  components,  or  parts  identified  as  critical. 
Detailed  and  specific  review  of  environmental  factors 
affecting  reliability  shall  be  performed.  In  addition  to 
qualification  and  acceptance  testing,  additional  testing 
shall  be  performed  on  critical  items,  such  as  life  testing 
or  failure-mode  testing,  to  assess  the  affects  of  the 
environments  on  such  critical  items,  and  to  determine 
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adequacy  of  safety  margins  incorporated  by  system  design, 
subject  to  approval  by  the  procuring  activity." 

"3. 5. 1.2  Component  Part  Testing.  All  component  parts  to  be 
used  in  production  equipment  shall  be  assigned  a  reliability 
index,  failure  rate,  or  expected  probability  of  failure  under 
stated  stress  levels.  The  reliability  test  procedures  of 
applicable  military  part  specifications  and  testing  spec¬ 
ifications  shall  be  used.  Where  the  contractor  deems  these 
test  procedures  not  applicable,  he  shall  submit  a  just¬ 
ification  of  non- applicability  and  a  description  of  the  test 
procedures  which  he  plans  to  use.  A  current  record  of  the 
results  shall  be  maintained.  The  test  data  shall  be  retained 
for  a  minimum  period  of  two  years  from  completion  of  contract. 
The  test  data  shall  be  made  available  to  information  and  data 
exchange  activities  upon  request  of  the  piocuring  activity." 

"3. 5. 1.3  Maximum  Preacceptance  Operation.  The  contractor 
shall  provide  and  maintain  a  current  list  of  items  having 
critically  limited  useful  lives  (total  operating  time  or 
operating  cycle)  in  their  application.  Derivation  of 
maximum  allowable  operating  time  for  cycles  of  operation) 
shall  be  clearly  defined  with  elements  of  data  and  methods 
of  computations.  The  contractor  shall  propose  ror  approval 
the  time  or  number  of  equivalent  operating  cycles  that  is 
not  to  be  exceeded  prior  to  acceptance  of  cho  contractor's 
product.  He  shall  ensure  that  each  such  item  has  its  total 
operating  time  or  number  of  equivalent  opera  ing  cycles 
recorded,  starting  with  and  including  its  initial  functional 
test,  whether  at  the  contractor's  facility  or  a  supplier's 
facility.  Upon  mutual  agreement  between  the  procuring 
activity  and  the  contractor,  any  item  may  be  dropped  from 
the  above  list,  or  its  limit  revised,  when  changes  in  the 
items  useful  life  indicate  the  need  for  such  revisions." 

"3.5.10  Statistical  Methods.  The  contractor's  reliability 
program  shall  incorporate  o'ptimum  utilization  of  statistical 
planning  and  analysis.  This  shall  include  application  of 
such  methods  as  design  of  experiment,  analysis  of  variance, 
and  other  methods  applicable  to  design,  development  and 
production  phases." 

The  basic  final  verification  of  achieved  reliability  is  the 
"sequential"  life  test,  which  is  usually  what  is  meant  by 
"reliability  demonstrations?" .  However  the  following  Mil-Std-785 
paragraphs  do  not  limit  reliability  testing  to  such  sequential 
life  tests : 
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"3.5.16  Reliability  Demons  tr  at  i'  >n 

"3.5.16. 1  Initial  Plan.  An  initial  plan  for  demonstration  of 
achieved  reliability  (and  maintainability)  at  specified  mile¬ 
stones,  including  estimated  number  of  test  articles  and  if  not 
specified  by  the  procu.  trig  activity  i  quantitative  estimate  of 
the  confidence  level,  shall  be  prepared  by  the  contractor  and 
submitted  in  a  section  of  the  reliability  program  plan.  The 
general  plans  for  demonstration  of  reliability  shall  include 
trade-off  curves  showing  number  of  test  articles  and  operating 
test  time  or  test  effort  versus  confidence,  and  will  encompass 
testing  at  the  system  major  element  level,  and  major  sub¬ 
system  or  component  levels  separately  and  in  combination. ’’ 

"3.5.16.2  Final  Plan.  Final  plan  for  demonstrating  achieved 
reliability  (and  maintainability)  shall  include  any  revisions 
to  data  in  the  initial  plan,  and  the  ground  rules  and  con¬ 
ditions  for  deciding  whether  a  test  shall  be  classified  as  a 
success  or  failure,  or  shall  be  excluded  due  to  invalid  test 
data.  Reliability  demonstration  plans  shall  apply  all  results 
of  testing  and  operations  from  which  valid  reliability  measure¬ 
ment  or  assessment  can  be  obtained.  Engineering  tests  and 
analysis,  e.g.,  test  to  failure  concepts,  shall  be  included 
to  supplement  statistical  measures.  The  milestones  that  are 
to  constitute  demonstration  of  contract  compliance  shall  be 
established  and  incorporated  in  the  contractual  documents. 
Specific  plans  for  conducting  a  reliability  demonstration 
shall  be  submitted  for  approval  at  the  time  specified  by  the 
procuring  activity." 

"3.5.16.3  Test  Plans.  Tte  test  plans  contained  in  (785  says 
Mil-Std-781,  but  BuShips  test  pl«ns  are  contained  in  Mil-R- 
22732B),  when  applicable,  shall  be  applied." 

But  often  reliability  life  tests  are  utterly  impractical  time- 
and  cost-wise,  so  verification  has  to  be  achieved  by  other  means. 
See  Chapter  11.  Since  testing  is  often  the  most  expensive  part 
of  a  reliability  program,  this  section  of  the  Program  Plan 
deserves  considerable  emphasis.  There  should  be  provision  for 
the  other  techniques  where  they  are  necessary.  And  simple 
economy  demands  some  kind  of  integrated  test  plan: 

"The  contractor  shall  completely  list  and  briefly  describe 
the  parts,  components ,  and  system  to  undergo  any  kind  of 
verification  of  reliability  or  maintainability.  Most  of 
these  will  be  part  of  a  larger  list  of  all  components  to  be 
tested,  included  in  the  Technical  Proposal,  and  should  be 


coded  thereto." 


"For  each  component  above  to  be  tested,  he  will  refer  to  a 
description  of  the  applicable  reliability  and/or  maintain¬ 
ability  test  procedure,  and  state  quantity  to  be  tested, 
test  criteria,  and  results  anticipated." 

"For  each  component  above  to  be  verified  by  othnr  than  test., 
he  will  briefly  state  the  rationale  for  such  decision,  the 
anticipated  analytical  procedure,  criteria,  and  results 
anticipated." 

"The  contractor  will  provide  a  chart  showing  the  time,  in 
relation  to  overall  design,  manufacture,  and  acceptance 
•test  schedules,  at  which  each  above  test  or  other  verification 
will  be  conducted." 


20-  SUMMARY 

In  this  chapter  we  have  described,  in  words  that  may  be  used  by 
the  Project  Engineer  to  the  contractor,  19  quite  basic  relia¬ 
bility  and  maintainability  tasks.  7i  comprehensive  program  on  a 
large  project  will  require  the  contractor  to  do  nearly  all  of 
them.  A  small  project,  or  a  parts  supplier,  may  need  to  do  only 
a  few  of  them.  The  determination  of  such  selection  and  depth 
is  covered  in  Chapter  23. 

Mil-Std-785  is  quoted  verbatim  herein  wherever  applicable  to  the 
19  tasks,  and  all  its  remaining  paragraphs,  since  they  do  not 
apply  to  specific  tasks,  are  quoted  in  Chapter  23.  It  will  be 
apparent  that  Mil-Std-785  must  be  supplemented  by  additional 
tasks  and  detail  wherever  a  comprehensive  program  is  needed 
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Chapter  23 
CONTRACT  PLANNING 

This  chapter  deals  with  the  broad  content  of  reliability  and 
maintainability  programs,  with  emphasis  upon  the  Project  Manage¬ 
ment  actions  necessary  to  plan  and  organize  such  programs  to 
mater  the  requirements.  The  Project  Management  actions  necessary 
to  conduct,  evaluate  end  control  such  programs  will  be  found  .in 
Chapter  24.  Detailed  treatment  of  specific  program  elements  will 
be  found  in  Chapters  3  t-hrough  22.. 

Throughout  this  chapter  and  Chapter  22,  the  MIL-STD-785  Specifi¬ 
cation  language  is  quoted  wherever  it  is  applicable,  and  supple¬ 
mented  by  other  language  as  needed.  Thus  these  two  chapters  ■ 
contain  all  such  selected  and  recommended  specification  language. 
However,  Chapter  17  contains  extracts  from  many  other  specifi¬ 
cations,  useful  as  a  reference  for  special  situations.  ^  s 

1.  REQUIREMENT  ESTABLISHMENT 

DOD  has  issued  (1)  rirective  3200.9  requiring  a  Project  Defini¬ 
tion  Phase  I  preceding  Acquisition  Phase  II  (for  actual  hardware 
development  and  production)  applicable  to  all  new  (or  major  modi¬ 
fication)  RDT&E  Engineering  Development  or  Operational  System 
Development  projects  over  $25  million,  or  requiring  production 
costing  over  $100  million,  PDP  Phase  I  results  in  a  PDP  Report 
which  actually  contains  tue  requirements  for  Phase  II.  Whether, 
or  not  PDP  is  applicable  to  a  given  BuShips  program,  the  require¬ 
ments  (2)  listed  in  Figure  23-4  express  the  DOD  point  of  view, 
and  can  serve  as  a  policy  for  any  program. 

The  purpose  of  any  reliability  and  maintainability  program  is  to 
actually  achieve  the  optimum  quantitative  level  of  reliability 
and  maintainability.  To  achieve  less  produces  inadequate  system 
effectiveness  and/or  excessive  maintenance  cost.  To  achieve  more 
may  require  excessive  contractor  program  costs.  Thus  it  is 
obvious  that  solid  requirements  must  be  established  before  it  is 
possible  to  determine  an  appropriate  program.  The  detailed  steps 
for  definition  of  requirements  have  been  given  in  Chapter  2. 

1,1  ENVIRONMENT  AND  STRESS 

The  BuShips  Technical  Code  should  first  thoroughly  establish  the  . 
conditions,  in  as  much  detail  as  feasible,  under  which  the  planned 
system  must  operate  for  all  probable  missions  during  its  use- 
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ful  lifetime-  The  Project  Engineer  should: 

"Completely  state  the  operational  environment  and  external 
stresses  acting  on  the  system,  by  mission  and  operational 
phase  or  mode,  for  the  anticipated  system  lifetime,  using 
stress  distributions  whenever  feasible. 

1.2  COST-EFFECTIVENESS  ANALYSIS 


See  Section  5.1  for  future  provisions. 

1.3  QUANTITATIVE  DESIGN  REQUIREMENTS 

Satisfaction  of  these  requirements  is  the  basic  objective  of 
the  entire  Reliability  and  Maintainability  Program  Plan.  The 
Project  Engineer  should  therefore: 

"State  the  required  minimum  acceptable  (mandatory)  and  opti¬ 
mum  reliability  (MTBF,  %  success  for  specified  mission  time, 
etc.),  maintainability  (MTTR ,  MAXTR,  manpower,  etc.)  and/or 
availability  {%  uptime  on  demand,  continuously,  etc.,  toget¬ 
her  with  the  test  acceptance  criteria  or  other  means  by 
which  each  will  be  verified  prior  to  BuShips  acceptance,  and 
wherever  feasible,  the  quantitative  required  confidence  levels." 

1.4  APPLICABLE  DOCUMENTS 


Several  documents  are  always  applicable  by  DOD,  Navy,  and  BuShips 
policy.  However  most  of  these  are  quite  broad,  so  that  some  in¬ 
dividual  sections  of  them  do  not  logically  apply.  Thus  it  is 
very  important,  in  the  interest  of  economy  and  to  avoid  the 
propagation  of  unnecessary  paperwork  and  confusion,  to  select 
the  applicable  sections.  The  Project  Engineer  should  therefore: 

"Based  on  the  actual  reliability  and  maintainability  values 
to  be  required  of  the  contractor,  and  taking  all  BuShips 
policies  into  account,  list  the  applicable  top  documents. 

For  each  document  specifically  list  the  section  numbers 
that  apply,  together  with  any  modification  of  a  section 
language  that  may  be  necessary.  Identify  which  are  mandatory 
and  which  subject  to  contractor  recommendation.  Provide  a 
file,  record  of  the  reasons  for  the  above." 

2.  PROPOSAL  MANAGEMENT 

So  far  as  reliability  and  maintainability  are  concerned,  this  may 
be  the  most  critical  phase,  deserving  the  greatest  attention. 
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For  it  is  a  fact  that  there  are  far  more  “lip  service"  relia¬ 
bility  programs,  even  among  otherwise  very  competent  contractors, 
than  there  are  bona  fide  reliability  achievement  programs.  Every 
possible  step  must  be  taken  to  make  sure  the  potential  contractor 
understands  what  he  is  getting  into. 

-.1  REQUEST  FOR  PROPOSAL 

Again  let's  review  in  Figures  23-7  and  23-8  what  the  DOD  view¬ 
point  is  for  RiD  over  $25  million.  We  continue  to  find  heavy 
emphasis  upon  definition  of  requirements,  encouragement  of  con¬ 
tractor  alternative  r  icommenda ^ions ,  and  the  balance  of  cost, 
schedule  and  performance  (i.e.,  effectiveness).  We  find  in¬ 
struction  to  advise  the  contractor  concerning  proposal  evaluation 
criteria  and  the  requirement  for  specification  of  quantitative 
reliability  and  maintainability. 

We  w'ill  concern  ourselves  only  with  those  elements  of  a  Request 
for  Proposal  (RFP)  or  Procurement  Request  that  affect  reliability 
and  maintainability,  since  all  other  elements  are  thoroughly 
covered  elsewhere.  In  each  of  the  following  sections  we  will 
give  a  bit  of  background,  followed  by  paragraphs  (in  quotes)  as 
examples  of  wording  that  BuShips  engineers  may  use  in  writing 
the  RFP. 

2*1*1  Requirements :  As  a  result  of  the  prior  definition  and 
analysis  work  outlined  in  Sections  1.1,  1.3  and  1.4  above,  the 
Project  Engineer  is  in  a  position  to  state  requirements  in  the 
RFP.  Should  he  temporarily  be  unable  to  specify  reliability  and 
maintainability  requirements,  the  contractor  should  nevertheless 
be  asked  to  state  what  be  will  achieve. 


2.1.2  Cost-Effectiveness  Analysis:  See  Section  5.1.1  for  future 
provis ions . 

2.1.3  Program  Plan:  This  principal  element  of  the  RFP,  Proposal 
and  resultant  contract  is  treated  separately  in  section  3*  and 
its  detailed  Task  Delination  in  chapter  22. 


2.1*4  Proposal  Due  Dates :  It  is  an  unfortunate  fact  that  many 
proposals  are  assembled  in  a  great  deal  of  pressure,  without 
adequate  time  for  analysis,  supplier  consultation,  and  thorough 
costing.  What  is  worse,  the  reliability  and  maintainbbility 
specialists  are  often  asked  to  "write  a  page  or  two,"  with  no 
opportunity  for  influencing  the  proposed  design  and  its  costing. 
The  reasons  are  largely  in  the  contractors  own  house,  and  the 
suggested  RFP  wording  in  this  chapter  should  get  attention  by 
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t* ! g  right  people. 

Under,  no  circumstances  should  the  knovvTi  desirable  bidders  be 
given  less  than  60  days  after  receipt  of  the  RFP  and  attendance 
at  the  bidders  conference.  If  a  thorough  analysis  is  required, 
such  as  the  cost-effectiveness  analysis  in  section  2.1.2,  it 
should  be  90  days.  Another  suggestion  is  to  provide  a  series  of 
due  dates,  requiring  the  analysis,  design,  and  management/ cost 
proposals  sequentially  30  days  apart. 

2.2 _ BIDDER  SELECTION  AND  CONFERENCE 

Of  course  bidders  arc  always  selected  from  among  those  known  to 
have,  the  technical  and  management  capability,  particularly  in¬ 
cluding  those  having  experience  with  similar  design  and  construct¬ 
ion.  The  following  is  quoted  {  5  )  from  DOD  Defense  Procurement 
Circular  dated  4  March  1964: 

“In  working  toward  better  defense  procurement,  nothing  is  more 
basic  to  satisfactory  procurement  than  that  we  deal  only  with 
responsible  prospective  contractors.  Contract  awards  to  con¬ 
cerns  of  marginal  capabilities  can  lead  only  to  delay's  or 
failures  in  obtaining  delivery  of  needed  items  and  to  increas¬ 
ed  eventual  costs  to  the  Government." 

"The  present  regulatory  guidance  on  responsible  prospective 
contractors  {A5PR  Section  I,  Part  9)  is  adequate.  It  is  not 
that  new  rules  are  needed,  but  that  our  present  rules  must  be 
understood  and  followed.  Importantly,  ASPR  requires  an 
affirmative  determination  that  the  prospective  contractor  is 
responsible  before  any  contract  award  may  be  made:  there  must 
be  a  posit ive  judgment  that  he  will  perform  the  contract  on 
schedule  in  accordance  with  its  terns. 

This  excludes  the  company  whose  qualifications  are  no  better 
than  borderline  as  to  production  capacity,  financial  capability, 
past  performance,  or  any  of  the  other  minimum  standards.  It 
excludes  the  company  whose  continuing  capability  throughout 
the  period  of  performance  is  jeopardized  by  a  pending  bankruptcy 
reorganization  or  other  evidence  of  financial  difficulty  which 
may  culminate  in  loss  of  needed  financial  capabilities  during 
the  period  of  contract  perform,  nee. 

It  means  that,  in  predicting  whether  a  company  will  perform 
the  contract  satisfactorily,  it  must  be  assumed  that  the 
Government  will  use  vigilant  and  forceful  contract  administra¬ 
tion.  It  is  not  acceptable  to  make  a  determination  of  res- 
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ponsibilit.y  which  envisions  completed  contract  performance 
only  after  extreme  Government,  financial  assistance  and  marked 
lenience  in  enforcing  delivery  schedules  or  other  contract 
terms . 

"Full  understanding  of  the  ^..iportance  of  affirmatively  deter¬ 
mining  that  the  prospective  contractor  is  responsible  should 
assist  our  efforts  to  increase  the  use  of  formally  advertised 
procurement.  Use  of  negotiation  is  never  justified  by  a  fear 
that  advertising  may  lead  to  aw’ard  to  a  contractor  who  is  un¬ 
likely  to  perform  satisfactorily.  The  standards  of  responsi¬ 
bility  for  contractors  are  precisely  the  same  for  advertised 
as  for  negotiated  procurements.  If  a  company  would  be  rejected 
as  not  responsible  notwithstanding  a  low  offer  in  a  negotiated 
procurement,  the  same  company  should  be  rejects-'  ■  otwithstanding 
^  low  Md  on  an  equivalent  advertised  procurement.  The  con¬ 
tracting  of  ficer  has  the  same  right  and  duty  to  determine  non¬ 
responsibility  in  one  case  as  in  the  other. 

"I  have  asked  the  Assistant  Secretary  of  Defense  (Installations 
and  Logistics)  to  take  the  necessary  steps  to  bring  the  import¬ 
ance  of  responsibility  determinations  to  the  attention  of  all 
contracting  officers. 


"Signed 

ROBERT  S.  M'  JAMARA 
Secretary  o.  Defense" 

The  recent  incorporation  of  quantitative  reliability  and  maintain¬ 
ability  requirements  certainly  does  not  change  these  very  sound 
selection  criteria.  On  the  other  hand  these  techniques,  and  the 
tasks  necessary  to  achieve  specified  reliability  and  maintain¬ 
ability,  are  new  to  many  BuShips  contractors.  Some  will  not  know 
what  "MTBF"  means.  They  will  havf>  many  questions,  and  there  is 
no  satisfactory  substitute  for  a  bidders  conference  at  wh  eh  the 
Bureau  design  and  reliability/maint.  inability  people  4  mtly 
respond  to  the  questions.  But  unless  the  contractors  reliability/ 
maintainability  specialist  is  specifically  invited,  the  contractor 
probably  won’t  send  one  even  when  he  does  ha  *e  such  competence. 

The  Project  Engineer  may  write  in  the  r'P: 

"The  contractor  will  send  questions  concerning  reliability 
and  maintainability,  in  writing,  to  (state  name,  address)  to 
arrive  no  later  than  (state  date).  He  will  then  send  one 
reliability/miintainabi lity  specialist,  as  well  as  other 
invited  representatives,  to  the  bidders  conference  to  be 
held  (state  date,  time  and  place)," 
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2.3  PROPOSAL  EVALUATION 


Background,  formats,  and  procedures  for  proposal  evaluation  are 
discussed  in  section  4. 

2.4  CONTRACT  NEGOTIATION 

This  is  a  large  subject.  We  are  concerned  here  only  with  those 
aspects  related  to  reliability  and  maintainability.  But  these 
aspects  are  so  serious  that  they  have  unquestionably  caused  much 
system  unreliability. 

2.4.1  Traditional  Approach;  Wherever  uie  design  techniques  are 
well-established,  a..d  all  contractors  know  very  well  just  what 
ehey  have  to  do  to  satisfy  the  stated  needs,  the  traditional  con¬ 
tractual  approach  works  fine.  BuShips  states  its  exact  require¬ 
ments,  the  contractors  bid,  and-  award  is  made  to  the  lowest  bidder 
who  has  the  necessary  capability. 

2.4.2  Advance  Development  Approach:  When  the  design  techniques 
are  not  yet  established,  and  no  capable  contractor  can  be  sure 
what  development  will  cost,  the  cost-plus- fixed- fee  approach  has 
been  widely  used.  However  it  has  been  abused  and  is  currently 
discredited.  But  it  does  recognize  the  basic  problem  of  con¬ 
tracting  for  uncertainty. 

Now  reliability,  in  its  youth,  reflects  a  design  uncertainty. 

Not  only  that,  it  is  an  uncertainty  of  much  greater  impact  on 
cost  and  effectiveness  than  many  typical  cost-plus  development 
uncertainties.  Small  wonder  that  it  is  difficult  to  get  realistic 
fixed  price  commitments  on  new  developments  with  guaranteed  relia¬ 
bility. 

2.4.3  Reliability  Incentive  Contracts:  Recent  DOD  instructions 
(6)  for  incentive  contracting  treat  Reliability  (barely)  as  part 
of  "performance",  (not  Performance  Capability),  and  Availability 
not  at  all.  They  do  not  reflect  recognition  of  the  problem  that 
has  arisen. 

There  have  been  a  few  contracts  that  adjust  fhe  percent  fee  in 
accordance  with  achieved  values  of  %  reliability.  Typically  such 
a  contract  has  paid  the  order  0.3%  added  fee  for  each  percent  of 
reliability  improvement  above  a  stated  target,  and  conversely 
below  it.  This  is  in  addition  to  the  cost-incentive  fee  that 
pays  10%  of  any  savings  below  a  target,  and  conversely  above. 
Figure  23-12  shows  how  it  works. 
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Unfortunately  the  cost  to  contractor  of  achieving  say  2%  higher 
reliability  is  much  greater  at  90%  than  70%.  The  difficulty 
and  cost  depends  directly  upon  how  close  the  reliability  is  to 
lo0%.  So  the  more  reliability  a  contractor  achieves  the  less  he 
is  paid  for  it.  It  works  out  so  that  the  contractor  is  actually 
penalized  if  he  tries  to  get  over  90%.  See  Chapter  26,  section 
7.8. 

2.4.4  Contracting  for  Cost-Effectiveness:  See  section  5.1.2  for 
future  provisions. 

2.4.5  Task  Negotiation:  Everv  proposed  task  should  be  analyzed 
tor  content,  output,  who  needs  it,  and  manpower.  "Who  needs  it" 
must  account  for  Bureau  as  well  as  contractor  needs.  Figure  23-14 
shows  how  reliability  can  be  substantially  affected  by  task  select¬ 
ion,  even  without  increasing  Acquisition  Cost,  but  that  actual 
design  for  specified  reliability  requires  additional  design  costs. 

The  ultimate  determination  whether  each  task  is  needed  should  be 
made  only  by  tr>«.  Engineer  on  sr...'.'/*:  ic  by  D-oui^s 

reliaoility/maintainability  specialists.  The  ultimate  decision 
concerning  proper  manpower  for  each  task  should  be  made  only  ■  / 
the  contractors  reliability/maintainability  representative. 

Negotiation  will  often  be  detrimental  to  reliability  and  cost- 
effectiveness  unless  both  are  present.  See  Chapter  24,  section  4. 

The  reliability  of  many  systems  has  been  degraded  by  negotiation 
decisions  made  by  people  unaware  of  the  consequences. 

2.5  CHANGE  PROPOSALS 

Change  proposals,  either  Ky  the  contractor  cr  BuShips,  should  be 
evaluated  identically  to  new  proposals.  It  makes  little  sense  to 
carefully  evaluate  the  original  proposal  and  make  decisions  vital¬ 
ly  affecting  reliability,  then  ignore  the  effect  of  altered  _ clia- 
bility  and  maintainability  in  changes. 

On  the  other  hand,  it  is  very  demoralizing  for  a  contractor  to 
propose  change  after  change  that  his  reliability  prediction  says 
ar*  needed  to  achieve  required  reliability,  only  to  have  them 
turneu  down  by  the  customer  for  reasons  like  "logistic  problems". 

This  keeps  happening. 

The  answer  would  appear  to  be  that  the  contractor  should  only  sub¬ 
mit  change  proposals  When  his  own  cost-effectiveness  analysis  says 
they're  worth  doing.  If  he's  right  BuShips  will  agree.  Or  at 
least  BuShips  should  us^  such  analysis  to  decide  on  their  dis- 
pos it  ion . 

i 
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3  PROGRAM  .-'LAM 

Adequate  and  economical  system  reliability  is  achieved  by  doing 
the  right  thing  at  the  right  time.  It  is  a  characteristic  wv loh 
cannot  be  imparted  at  any  one  ^1..*.  in  the  program,  but  >  hich 
can  be  destroyed  by  one  error  of  omission  or  To 

control  system  reliability,  t:  *n,  the  reliability  consequence  of 
every  design  decision  in  system  and  hardware  development  must  be 
clearly  recognized  and  pro  vision  mad*  to  get  the  right  result . 

This  implies  a  level  of  detail  in  technical  and  program  manage¬ 
ment  wh  1  cii  is  very  difficult  to  achieve  in  the  traditional, 
comm,  uni  o  at  l  on  network  of  industrial  management. 

The  f  irmal  Program  Plan,  therefore,  has  fa-  en  found  necessary  for 
management  (a)  recognition  t  the  real  requirements ,  (b)  assur¬ 

ance  that  the  necessary  tasks  are  planned,  scheduled,  funded  and 
accomplished  on  time,  (c)  technical  visibility  of  the  ultimate 
consequence  of  alternative  design  d*‘cis  ions ,  and  (;i)  visibility, 
throughout  design,  of  the  probable  result  vs.  requirements .  With¬ 
out  such  a  plan  reliability  achievement  is  very  expensive  and  in- 
e  f  feet.  n  o. 

The  reliability  and  maintainability  of  a  design  is  only  what  the 
ies  ign  engineer  puts  there.  Tt’s  'v**y  trite,  but  very  true.  Yet 
the  reliability  technology  has  be- come  s  ■  complex,  and  the  litera- 
t  ui e  so  voluminous ,  that  tiu  aver  i  :e  design  engineer  cannot  pos¬ 
sibly  learn  all  of  it  and  still  bo  a  good  design  engineer.  So 
industry  ha-:  had  to  dove  lop  reliability  and  mnint  ainabi  1  ity 
specialists  to  " support "  the  dvsi-.n  engineer.  But  reliability 
support  is  m  utter  •  iste  of  money  it  it  is  not  an  integral  pat  t 
of  the  desian  or  'gram,  and  early  >  nough  f-T  results  to  be  used. 
This  can  be  aoeompl  i.,;.od  only  by  dot  ailed  planning  based  on  in¬ 
timate  knowledge  of  the  vies  i  an. 


The  Reliability  an-i  Maintainability  (or  "Dependability"!  Program 
Plan  shoul-,1  be  the  single  instrument,  of  detailed  understanding 
as  to  exact ly  what  will  be  accomplished  t  achieve  required  relia¬ 
bility  and,  m  a  i  nt  a  i  nab  i  lit  y  .  Initially  it  constitutes  Bureau 
agreement  with,  higher  authority.  Then  it  becomes  the  Bureau 
agreement  with  the  contractor.  Finally  it  is  the  contractors 
cvnpreh**ns  ive  statement  ot  his  understanding  of  the  problem,  what 
he  will  d  *,  how  he  will  do  it,  when  he  will  ctnplete  each  item, 
and  what  m.mpvwvr  it  t  ikes . 

Reliability  program  plans  have  been  suite  th-  roughly  developed 
(  h  by  man-  contract  ors  over  f!io  past  r'  to  l  1  years .  Maintain¬ 
ability  pivKiram  plans  have  been  only  briefly  developed,  and  they 


; 

i 

t 
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essentially  parrot  conventional  reliability  program  plans.  T bert¬ 
has  been  some  effort  (9)  to  combine  the  two,  but  without  much 
cor.so  1  ida t ion  of  tasks  common  to  both.  The.  following  treatment 
cover:;  both,  and  wherever  one  task  will  accomplish  both  it  is  so 
stated.  This  will  minimize  paperwork,  cost,  Qnd  confusion. 

While  BuShips  may  wish  to  add,  delete,  or  alter  many  items  of  the 
contractors  proposed  plan,  there  absolutely  m u s t  be  agreement  on 
its  content  and  meaning  before  work  starts.  Keeping  in  mind  that 
we  have  added  the  11  (a  id  maintainability) "  wherever  it  is  equally 
applicable,  MIL-3TD-78 5  states: 

"3.1  Reliability  (and  maintain abi lity)  Assurance  Program. 

The  contractor  shall  establish  and  maintain  an  effective  and 
economical  reliability  fand  maintainability)  assurance  program, 
planned,  integrated,  ^nd  developed  in  conjunction  with  other 
planning  functions.  The  program  shill  be  adjusted  to  suit  the 
type  and  (design,  development,  or  production)  phase  of  the 
procurement.  The  program  shall  be  based  upon  the  severity  of 
the  requirements,  the  complexity  of  the  design,  the  quantity 
under  procurement ,  and  the  manufacturing  techniques  required. 
The  program  shall  as  ;ure  adequate  reliability  fand  maintain¬ 
ability)  consideration  throughout  all  aspects  of  the  design, 
development,  or  production  as  necessary  to  meet  the  contract¬ 
ual  reliability  (and  maintainability)  requirements." 

"3.3  Reliability  (and  Maintainability)  Plan." 

"3.3.1  Proposed  Reliability  (and  Maintainability)  Plan.  The 
contractor's  proposed  reliability  (and  maintainability'  pro¬ 
gram  pxan,  in  accordance  with  the  requirements  of  the  work 
statement  and  this  standard,  shall  be  submitted  as  a  separate 
and  complete  entity  within  the  contractor's  proposal  for  the 
system.  The  proposed  plan  must  be  an  integrated  effort  within 
the  total  program  plan;  it  shall  provide  specific  information 
as  to  how  the  contractor  will  meet  specified  quantitative 
reliability  (and  nnuintainab  1  ity )  requirements  during  develop¬ 
ment  anc.  manufacture  including  the  design  concepts  to  be 
utilized.  The  proposed  manner  of  demonstrating  reliability  at 
stated  confidence  levels  shall  be  described.  The  proposed 
reliability  (and  maintainability)  program  plar,  as  approved  by 
the  procuring  activity  will  become  a  contract  compliance  docu¬ 
ment;  re^abilitv  (and  maintainability)  test  plans  must  be  an 
inteoral  part  of  the  program  test  plan." 


In  order  to  encourage  uniformity  across  proposals  to  permit  valid 
evaluation,  the  Project  Engineer  may  add  these  words: 
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"This  Plan  shall  contain  sections  cn  (a)  Requirements,  (b) 
Program  Charts,  (c)  Program  Tasks,  (d)  Organization,  and  (e) 
Competence,  as  detailed  below.  The  content  of  every  section 
must  state  primarily  (a)  what  will  be  done  to  be  exactly  res¬ 
ponsive  to  BuShips  stated  requirements,  then  (b)  the  con¬ 
tractors  alternative  recommendation,  if  any,  and  then  invari¬ 
ably  (c)  what  portion  tne.  euf  is  already  in  contractor  opera¬ 
tion  or  has  been  done  on  past  stated  contracts." 

"If  the  contractor  is  successful,  all  proposed  sections  {ex¬ 
cept  Competence)  will  be  negotiated  in  detail  and,  upon  agree¬ 
ment,  will  become  the  contractual  Program  Plan.  Though 
it  may  later  be  modified  by  mutual  agreement,  it  is  the  sole 
contractual  statement  of  reliability  and  maintainability  work 
and  results  to  be  obtained." 

The  necessary  content  of  the  Program  Plan,  and  especially  the 
Program  Tasks  included,  will  vary  widely  depending  upon  (a)  the 
gap  between  needed  and  available  reliability  and  maintainability, 
(b)  how  well  the  available  reliability  and  maintainability  is 
known,  and  (c)  the  design  level  (system  vs.  parts).  Figure  23-18 
shows  a  recommended  "starting  point"  for  a  very  comprehensive 
program,  but  actual  selection  of  tasks  must  rest  on  the  specific 
situation.  The  table  applies  only  to  "critical"  reliability  and 
maintainability  items  as  defined  in  Chapter  27.  However,  there 
may  be  many  items  of  such  low  "criticality"  (system  failure  rate 
increment  due  to  that  item)  relative  to  others  that  such  control 
of  ther  may  not  be  cost-effective.  "Criticality",  incidentally, 
is  a  numerical  value  on  a  continuous  scale  roughly  equivalent  to 
the  stepped  "levels  of  essentiality"  and  "classification  of 
characteristics"  used  in  quality  control  techniques  where  a 
prediction  mode]  is  not  available. 

In  an  activity  as  urgent  as  that  of  reliability  achievement,  a 
tendency  toward  over-emphasis  often  occurs.  Since  much  of  the 
effort  in  such  programs  is  support  rather  than  design,  its  appli¬ 
cation  to  unnecessary  areas  can  incur  significant  unneeded  expense 
Each  Program  Task  must  be  weighed  carefully  in  terms  of  its  con¬ 
tribution  to  cost-effectiveness.  Where  Navy  experience  shows  tniu. 
traditional  design  practice  or  construction  methods  produce  trou¬ 
ble-free  hardware  -  such  as  perhaps  is  the  case  with  internal  bulk 
heads,  for  example  -  extensive  reliability  assurance  activity  is 
not  warranted. 

Even  in  areas  of  the  design  where  there  is  some  concern  over 
reliability,  task  applicability  will  vary  with  the  type  of  hard¬ 
ware.  While  great  care  may  be  needed  in  receiving  inspection 
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and  process  control  for  structural  materials,  with  little  emphasis 
on  sophisticated  reliability  analysis,  these  latter  techniques  may 
prove  most  effective  in  disclosing  opportunities  for  reliability 
improvement  of  electrical  power  supplies. 

3 • 1  REQUIREMENTS 

Reliability  requirements  are  the  foundation  of  every  Program  Plan. 
To  d^te,  the  defense  procurement  record  is  bleak  in  terms  of  relia¬ 
bility  requirements  definition.  Studies  updated  as  recently  as 
February  1964,  indicate  that  less  than  one  third  of  the  system 
development  programs  in  which  reliability  is  stated  to  be  an 
important  factor  speci fy  this  requirement  adequately.  The  result 
is  that  system  design  and  manufacture  either  ignore  or  grope 
around  the  edges  of  reliability  technology,  with  neither  customer 
nor  contractor  possessing  a  yardstick  to  measure  achievement 
against.  As  in  any  activity,  reliability  and  maintainability 
achievement,  and  the  Program  Plan,  degenerate  to  pure  motherhood 
and  sin  without  quantitative,  measurable  goals. 

The  requirements  stated  in  the  Dependability  Plan  must  be  pre¬ 
faced  by  adequate  operational  information.  This  includes  such 
items  as  planned  deployment ,  reaction  time  required,  duration  of 
each  kind  of  mission,  turn-around  time,  overall  mission  relia¬ 
bility,  availability,  combat  ready  rate,  environmental  conditions, 
and  planned  utilization  rate.  The  broad  operational  maintenance 
philosophy  and  policy  must  be  stated. 

Although  the  BuShips  reliability  and  maintainability  requirements 
will  have  been  stated  to  the  contractors  in  the  RFP ,  the  contractor 
should  be  required  to  state  them  back  in  the  Program  Plan,  with 
recommended  modification,  if  any.  The  objective  is  to  make  very 
sure  of  common  understanding  and  agreement  on  the  baseline  upon 
which  all  tasks  are  based.  See  chapter  2.  MIL-STD-785  states: 


“(3. 5. 1.1)  If  maximum  environmental  stress  conditions  have 
not  been  established  by  the  procuring  actnity  these  shall 

be  estimated  from  experience  on  past  programs . Detailed 

and  specific  review  of  environmental  factors  affecting 
reliability  shall  be  performed." 

Since  environment  and  operational  stress  have  a  major  influence 
on  reliability  achieved,  the  Project  Engineer  may  wish  to  add: 

"The  contractor  shall  evaluate  the  BuShips  statement  of 
operational  environment  and  stresses,  and  recommend  any  specific 
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modifications  thereof  that  in  his  own  experience  .are  appro¬ 
priate,  However  the  proposal  must  be  based  upon  a  least  meeting 
stated  environments  and  stresses." 

Concerning  quantitative  requirements,  MIL  STD  785  states: 

"3.2.1  Quantitative  Requirements,  The  system  reliability 
objectives  and  minimum  acceptable  requirements  shall  be  as 
specified  contractually.  The  minimum  acceptable  reliability 
requirements  for  some  major  subsystems  and  equipments  may  be 
included  in  appropriate  sections  of  the  system  specification. 

The  values  not  established  by  the  procuring  activity  shall  be 
established  by  the  system  contractor  at  a  contractually  speci¬ 
fied  control  point  prior  to  release  of  design  for  initial 
fabrication  of  specified  articles." 

The  Project  Engineer  may  wish  to  add: 

"The  contractor  shall  evaluate  the  BuShips  statement  of  min¬ 
imum  (mandatory)  and  optimum  quantitative  reliability  and 
maintainability  requirements,  and  recommend  any  specific- 
modifications  he  cares  to  make.  However,  the  proposal  must 
be  based  upon  meeting  the  stated  optimum  requirements." 

Concerning  documents,  MIL  STD  785  states: 


■  .  | 

ol 

..if 


Referenced  Documents 


MIL  STD  721A 
MIL  STD  7  56A 
MIL  STD  781 


MIL  HDBK  217 


MIL  STD  803 


Definitions  for  Reliability  Engineering. 

Reliability  Prediction. 

Test  Levels  and  Accept/P* ject  Criteria  for 
Reliability  of  Nonexpendable  Electronic 
Equipment . 

Reliability  Stress  and  Failure  Rate  Data 
for  Electronic  Equipment 

Human  Engineering  Criteria  for  Aircraft, 
Missile,  an-  Space  Systems,  Ground 
Support  Equipment." 


-  ■' f  , 


Since  the  applicability  of  the  above  documents  to  most  specific 
proposals  will  be  very  spott^  indeed,  the  Project  Engineer 
should  add  the  appropriate  paragraph  numbers.  He  may  also  state: 


"The  contractor  shall  acknowledge  the  BuShips  list  of  speci¬ 
fications,  standards,  performance  specifications,  work  state¬ 
ments,  etc.,  and  their  required  section  numbers,  in  the  Pro¬ 
gram  Plan.  Under  each  the  contractor  shall  state  (a)  the 
required  section  numbers  that  are  acceptable,  (b)  those  for 
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which  modification  is  recommended ,  and  the  reason,  and  (c) 
additional  sections  recommended,  and  the  reason.  Then  he 
shall  add  any  other  documents  upon  which  his  proposal  is 
based,  calling  out  applicable  sections  thereof  and  providing 
copies . " 

3.2  PROGRAM  CHARTS 
According  to  MIL  STD  785: 

"3.3.3  Management  and  Control.  The  program  plan  shall  in¬ 
clude  detailed  listing  of  specific  tasks,  man-loading  per 
task,  and  procedures  to  implement  and  control  these  tasks. 

It  shall  include  a  description  of  each  task  to  be  performed 
whether  or  not  it  is  already  documented  in  contractor  direc¬ 
tives,  the  organizational  unit  with  the  authority  and  respon¬ 
sibility  for  executing  each  task,  the  method  of  control  to 
insure  execution  of  each  task  as  plan. ted,  and  scheduled  start 
and  completion  dates  of  each  task.  This  data  shall  be  in  a 
form  that  permits  technical  auditing  by  the  procuring  activ¬ 
ity.  The  information  provided  shall  include  the  method  of 
analysis  to  be  used  as  a  basis  for  achieving  the  proper  Dalance 

of  effort  and  resources  from  a  reliability  standpoint . The 

designation  of  milestones,  definition  of  inter-relationships , 
and  estimation  of  times  required  for  reliability  program 
activities  and  tasks  shall  be  employed  as  part  of  overall 
program  control  which  applies  the  program  techniques.  If 
PERT  (Program  Evaluation  and  Review  Techniques)  is  part  of 
the  program  it  shall  be  utilized." 

In  order  to  provide  a  practical  index  of  all  reliability  and 
maintainability  Program  Plan  tasks,  as  well  as  an  auditable 
summary  of  their  schedule  and  progress,  the  Bureau  may  develop 
and  specify  a  management  presentation  system  such  as  the  follow¬ 
ing  : 

3.2.1  Task  Number  and  Title:  In  Figure  23-22,  all  Tasks  dis¬ 
cussed  in  chapter  22  have  been  listed  and  given  an  index  letter 
for  easy  reference.  The  Project  Engineer  may  specify  in  the  RFP: 

"The  contractor  shall  provide  and  maintain  a  one-  or  two-page 
Task  Identification  chart  which  lists  the  title  of  every  Pro¬ 
gram  Task  to  be  undertaken,  preceded  by  a  reference  letter  or 
number  thenceforth  always  used  to  designate  that  task.  Tasks 
shall  be  listed  in  the  approximate  sequence  to  be  undertaken, 
as  in  Figure  23-22." 


TASK  IDENTIFICATION 


Verification  ]  392-476  85  Approved  Verifications 
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3.2.2  Document  Sections;  Since  many  specifications  contain 
overlapping  requirements,  this  chart  provides  the  cross-index  of 
multiple  call-out,  as  well  as  easy  reference  to  specification 
section  numbers: 

"The  contractor  shall  provide  a  column  for  each  major  document 
that  requires  reliability  or  maintainability  tasks,  listing 
in  each  the  docuu.  snt  section  numbers  applicable  to  each  task." 

3.2.3  Output :  Unless  the  output  or  tangible  end-product  of 
each  task  is  clearly  identified,  there  is  no  way  for  either  Bu- 
Ships  cr  the  contractor  to  audit  accomplishment.  This  require¬ 
ment  is  intended  to  neutralize  Parkinson's  third  law.  it  is  not 
so  uncommon  to  find  groups  performing  tasks  whose  output,  if  any, 
nobody  needs : 

"The  contractor  shall  provide  for  each  task  (a)  the  name  of 
its  principal  output  (identifiable  end  result),  (b)  the 
number  of  such  outputs  to  be  produced,  and  (c)  a  unique  out¬ 
put  number  range  assignment  for  schedule  identification.  If 
there  is  more  than  one  significant  kind  of  output,  another 
line  may  be  used.  The  Task  Delineation  (chapter  22)  must 
state  who  uses  each  output." 

3.2.4  Res pons ibility :  One  of  the  prime  problems  in  many  re¬ 
liability  programs  is  undetermined,  contested  or  weak  responsi 
bility  and  authority,  leading  to  redundant  costs  and  loss  of 
good  people.  One  way  to  cause  contractor  management  to  face  up 
to  unequivocal  decision  is  to  require  clear  statement  in  the 
reliability  and  maintainability  Program  Plan  of  exactly  who  is 
responsible  for  each  task  and  gets  replaced  if  it  doesn't  go 
well.  It  also  tells  BuShips  who  to  put  the  finger  on. 

"The  contractor  shall  provide  a  column  showing  by  code  lettars 
(such  as  the  organizational  group  name  initials)  exactly 
which  one  qroup  has  responsibility  and  authority  for  the  tank, 
and  which  other  groups  will  expend  support  manhours." 

3-2.5  Manpower :  Relative  effort  to  be  applied  to  each  task 
provides  an  important  clue  to  contractor  understanding  of  work 
content,  and  of  course  is  necessary  during  negotiation  to  fix 
exact  task  content.  And  sometimes  the  cost  of  reliability 
achievement  will  be  an  eye-opener  to  BuShips  personnel,  but  must 
be  balanced  via  cost-effectiveness  analysis  against  the  potential 
maintenance  manpower. 

"The  contractor  shall  provide  a  column  showing  the  total 
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manweeks  required  to  produce  all  outputs  for  each  task  for 
the  entire  program." 

3.2.6  Task  Schedule:  During  the  proposal  phase  this  provides 
some  evidence  that  the  contractor  has  thought  through  the  timing 
of  each  task  output  relative  to  the  design  and  production  sche¬ 
dules.  As  design  gets  underway  it  provides  a  convenient  means  of 
visualization  and  audit  of  conc-actor  progress. 

"The  contractor  shall  provide  on  a  separate  page  a  Task 
Schedule  chart  such  as  Figure  23-25  with  a  column  of  short 
task  titles  and  their  numbers  in  the  same  sequence  as  in 
3.2.1  above,  with  week  numbers  across  the  page  top,  identi¬ 
fying  key  contract  weeks  by  date.  The  chart  will  show  under 
each  week  number,  opposite  each  task,  the  task  start  week(s) 
and  output  numbers  as  they  will  fall  due,  all  su>.n  due  dates 
being  events  that  must  be  consistent  with  the  overall  PERT/ 
Cost  network." 

3.3  TASK  DELINEATION 


A  comprehensive  reliability  and  maintainability  program  for  a 
large  program  contains  a  lew  uozen  fa;rlv  distinct  Tasks,  the 
principal  ones  being  listed  in  chapter  22.  Lesser  emphasis  and/ 
or  smaller  projects  require  fewer  elements  and/or  less  depth  ''f 
effort  withir.  the  elements.  The  contractor  should  be  encouraged 
to  state  his  own  estimate  of  the  tasks  and  depth  needed  to  satis¬ 
fy  the  stated  requirements,  yet  always  show  adequate  considera¬ 
tion  of  the  tasks  listed  in  chapter  22.  Again  referring  to  MIL 
S^D  785: 

'3.3.3  Management  ar.d  Control.  The  program  plan  shall  in¬ 
clude  detailed  listing  of  specific  tasks,  man-lo3ding  per 
task,  and  procedures  to  implement  and  control  these  tasks  . 

It  shall  include  a  description  of  each  task  to  be  performed 
whether  or  not  it  is  already  documented  in  contractor  direc¬ 
tives,  the  organizational  unit  with  the  authority  and  respon¬ 
sibility  for  executing  each  task,  the  method  of  control  to 
insure  execution  of  each  task  as  planned,  and  scheduled  s  t  a,  ; 
and  completion  dates  of  each  task.  This  data  shall  be  in  a 
form  that  permits  technical  auditing  by  the  procuring  activity. 
The  information  provided  shall  include  the  method  of  analysis 
to  be  used  as  a  basis  for  achieving  the  proper  balance  of 
effort  and  resources  from  a  reliability  standpoint . . . .The 
designation  of  milestones,  definition  of  inter-relationships, 
and  estimation  cf  times  required  for  reliability  program 
activities  and  tasks  shall  be  employed  as  part  of  overall 
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program  control  which  applies  the  program,  techniques.  If 
PERT  (Program  Evaluation  and  Review  Techniques)  is  part  of 
the  Program  it  shall  be  utilized." 

Thus  in  relation  to  the  contractors  tasks,  the  Project  Engineer 
may  specify: 

"For  each  of  the  Tasks  listed  in  (chapter  22) ,  the  contractor 
shall  (a)  state  whether  or  not  effort  cn  this  task  is  planned, 
and  if  not  why  not;  and  if  planned  (b)  describe  concisely 
how  the  task  will  be  accomplished,  showing  the  depth  of  detail 
anticipated,  (c)  svate  what  tangible  or  recognizable  and 
auditable  output  will  be  produced,  and  (d)  state  what  specific 
organizational  groups  will  use  the  output  and  for  what  purpose. 
Do  not  use  more  than  one-half  to  one  page  per  task." 

"Then  the  Program  Chart  (section  3.2  above)  shall  show  (e) 
what  document  sections  call  it  out,  (f)  what  groups  art  re¬ 
sponsible  for  doing  and  supporting  the  work,  (g)  what  manpower 
(manweeks)  is  required,  and  (h)  when  each  successive  output 
is  scheduled  duo." 

3.4  ORGANIZATION 

While  BuShips  cannot  and  should  not  dictate  the  contractors  in¬ 
ternal  organization,  it  is  nevertheless  often  a  very  -important 
indication  of  his  appreciation  of  the  reliability  and  maim  a  in¬ 
ability  problem,  and  therefore  his  competence.  See  reference 
(g,  secs.  4,5,6).  MIL  STD  ?85  states: 

"3.3.2  Peliabi lity  (and  maintainability!  Organ izat ion.  The 
program  plan  shall  (a)  identify  the  organ i ? at  ion  and  the  per¬ 
sonnel  responsible  for  managing  tne  overall  reliability  (and 
maintainability)  program,  and  (b)  shall  clearly  do! ine  its 
repsons ibi i it i es  and  funet ions  -no lading  both  policy  »nd 
action.  It  shall  stipulate  the  authority  delegated  to  this 
organ i /at i on  to  enforce  its  policies.  Thu  relat ionships 
between  lire,  service,  at  iff,  and  policy  organ izat  ions  shall 
be  l dent  i  t  ied  .  " 

3.4.1  C  an  p  a  n  y  a t ructur e  ;  Here  the  objective  is  to  f ind  out 

whether  reliability  and  ma i nt a inabi 1 i ty  j roups  (a)  actually 
exist,  and  (b)  are  close  enough  to  design  engineers  to  be  effec¬ 
tive,  or  whither  they  are  'ivory  tower"  generators  of  procedures 
and  reports  that  nobody  r^ads  and.  on  which,  nobody  takes  action. 
Although  such  central  groups  are  usually  called  "Reliability 
Engineering,"  there  is  a  st  ror.  trend  toward  combining  such 
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activities  and  calling  them  "product”  effectiveness,  system 
effectiveness,  or  (recently)  cost-effectiveness.  In  order  to  get 
proposal  uniformity  for  evaluation  purposes  the  Project  Engineer 
may  speci fy : 

"The  contractor  shall  provide  an  organization  chare  showing 
(a)  the  major  functional  and  project  groups  reporting  to  uhe 
top  executive,  and  (b  a  line  of  authority  breakdown  to  those 
engineering,  reliability,  and  maintainability  groups  that  will 
directly  contribute  to  the  program.  The  latter  must  show 
every  supervisory  level  in  the  hierarchy,  identifying  the 
working  groups  phe  order  of  3  to  20  people)  with  code  letters 
or  numbers  called  out  in  the  Program  Chart  (section  3.2.4) 
responsibility  column.” 

"Each  organisational  block  must  show  the  name  of  the  group, 
the  name  of  its  supervisor,  and  the  total  number  of  its  per¬ 
sonnel,  using  parentheses  around  each  planned  but  not  existing 
entry . 

3.4.2  Policy  Direction:  Not  much  gets  accomplished  until  the 
top  executive  or  his  top  subordinates  issue  directions  in  un¬ 
equivocal  and  clear  terms.  Such  directives  are  usually  signed 
by  the  top  executive  and  state  what  is  to  be  accomp lished ,  by 
whom,  and  how  he  verifies  ♦'hat  it  gets  done.  The  Project  Engin¬ 
eer  may  specify: 

"The  contractor  sha1 1  provide  'py  or  quotation  of  the  re¬ 
liability  and  maintainability  policy  directives  and  standard 
procedures  binding  upon  all  engineering,  reliability  and 
maintainability  personnel,  with  signature,  title,  and  date. 

He  shall  include  both  company-wide  directives  and  procedures 
and  those,  if  any,  concerning  this  program,  and  state  how  he 
assures  that  they  are  kept  current,  and  complied  with." 

3.4.3  Responsibility  and  Authority:  The  design  engineering 
groups  must  have  the  final  decision  on  oil  design  decisions,  or 
else  they  could  not  be  held  accountable  for  their  designs .  Yet 
there  must,  be  procedures  whereby  such  decisions  cannot  be  made 
until  there  is  adequate  consideration  of  reliability  and  main¬ 
tainability  impact.  In  most  cases  the  design  engineer  is  too 
close  to  his  design  to  be  a  good  judge  of  such  adequacy ,  so  the 
reliability  and  maintainability  groups  must  review  every  new 
design  and  make  recommendations .  if  any. 

If  the  design  decision  is  contrary  to  such  a  recommendation, 
which  should  be  rare,  there  must  be  independent  channels  by  which 
the  rccor.mer.dat  ion  can  be  taken  up  the  hierarchy  for  recons  icier  a- 
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tion,  even  by  the  top  executive  in  rare  cases.  For  comprehensive 
programs  the  Project  Engineer  may  specify: 

"Referring  to  the  Organization  Chart  (3.4.1),  the  contractor 
shall  briefly  list  the  responsibilities  and  authorities  assigned 
to  design  vs.  each  reliability  and  maintainability  group  in 
matters  concerning  reliability  and  maintainability  achievement , 
analysis ,  and  verified! .on.  The  contractor  shall  explain  the 
flow  of  work  between  such  groups,  and  state  how  differences  of 
opinion  are  re?  lived.  Fie  will  specifically  list  the  documents 
that  require  reliability/maintainability  group  (a)  review, 
and  (b)  signoff. 

3.4.4  Program  Control:  Unless  a  specific  group  has  the  respon¬ 
sibility  for  keeping  track  of  task  performance,  keeping  task  pro¬ 
gress  in  balance,  and  directing  relative  task  effort  accordingly, 
schedules  will  certainly  slip  and  reliability  suffer.  The  Project 
Engineer  may  wish  to  be  more  specific: 

"The  contractor  shall  name  the  group  responsible  fer  reli 
bility/maintainabi.lity  program  control,  and  state  the  proce¬ 
dures  it  will  use  to  keep  task  progress  in  balance  and  on 
schedule,  and  to  direct  relative  effort  apportionment  thereon." 

3 . 3  COMPETENCE 

Often  a  contractor  will  seem  to  thoroughly  understand  the  problem, 
and  wi.il  propose  an  excellent  program  plan,  yet  make  many  costly 
and  schedule-slipping  mistakes  before  producing  the  required  re- 
. iability  and  maintainability.  There  is  no  substitute  for  ex¬ 
perience,  and  the  RFP  must  require  the  proposal  to  provide  quite 
sped,  f  i  c  answers  . 

3.5.1  i  im-gr ams  :  Until  a  contractor  has  had  to  design  to  spec¬ 
ified  reliability  and  maintainability,  ne  usually  considers  it  a 
big  propaganda  and  quality  control  drive.  In  order  to  ferret 
out  exactly  wh.ic  a  contractor  has  accomplished  the  Project  Engin¬ 
eer  may  specify: 

"The  contractor  shall  list  the  programs  he  has  undertaken 
which  contractually  required  (a)  design  to  specified  relia¬ 
bility  and/or  maintainability  quantitative  values,  (b)  design 
to  "high"  reliability  requirements,  or  (c)  design  and  produc¬ 
tion  with  execution  of  specific  reliability  Program  Plan  tasks. 
For  each  the  contractor  will  state,  if  known,  how  the  achieved 
quantitative  reliability  and,/ or  reliability  compared  to  values 
predicted  in  the  initial  Program  Plan." 
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3  . "  .  2  Technology  Development. :  M  my  progress  :  ve  companies  de- 
•  ..  !  .  4->'*-'tr  reliability  and  maintainability  technical  capability 

by  fondina  long-term  research  projects  or  by  funding  shorter- 
term  projects  that  •••■'ill  ch  tllenge  and  hold  their  best  people 
between  programs .  To  icier.t  i  fy  this  latent  capability,  the 
Project  Engineer  may  specify: 

"The  contractor  shall  list  and  briefly  describe  the  reliabil¬ 
ity  and/or  maintainability  technique  research  or  development 
projects  he  has  undertaken  either  on  contract  or  company 
funding,  and  for  each  the  actual  application  of  results." 

3.5.3  Industry  Participation:  The  extent  of  participation  in 
government  and  industry  committees  and  conferences  is  also  an 
indicator  of  company  appreciation  of  the  problem  and  contribu¬ 
tion  to  its  solution.  The  Project  Engineer  may  specify: 

"The  contractor  shall  list  the  reliubi1 ity  and/or  maintain¬ 
ability  government  and  industry  committees  on  which  he  is 
represented,  and  for  each  the  name  and  company  pcition  of 
his  representative.  The  contractor  shall  also  list  the  re¬ 
liability,  maintainability,  system  effectiveness,  and  cost- 
effectiveness  papers  and  conference  sessions  prepared  or 
moderated,  by  his  personnel  during  the  previous  year,  and  for 
each  the  person's  name  and  position." 

3.5.4  Resumes :  Most  proposal  resumes  do  not  indicate  the  ex¬ 
perience  that  personnel  may  have  had  in  des ign  for  reliability 
and  maintainability,  t. -sough  analyses  experience  is  covered.  The 
Project  Engineer  may  specify: 

"The  contractor  shall  provide  one-third-page  resumes  on  each 
of  the  key  personnel  that  will  be  responsible  for  achievement 
of  required  reliability  and  maintainability.  These  should 
include  design  as  well  as  reliaoility  and  maintainability 
people,  stating  their  reliability  and  maintainability  design, 
analysis,  and  supervisory  experience.” 

3.5.5  Task  Experience.  Nearly  all  proposals  will  contain  pro¬ 
posed  tasks  that  the  contractor  has  never  done  before.  Often  as 
not  he  may  consider  them  as  unnecessary  embel lishnents ,  but  of 
course  wants  to  he  strictly  responsive  to  the  RFP.  Since  prior 
experience  has  a  great  effect  on  his  understanding  and  support 
of  the  task,  there  must  be  some  way  to  identify  such  task  exper¬ 
ience.  As  stated  in  section  3.,  all  Task  delineations  in  chap¬ 
ter  22  must  state  what  portion  thereof  is  already  in  contractor 
operations  cr  has  been  done  on  past  stated  contracts. 
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3.5.6  Supplier  Competence:  It  is  one  thing  to  propose  an  ex¬ 
cellent  program  with  tasks  that  subcontractors  and  vendors  are 
expected  to  accomplish,  but  quite  another  to  get  some  of  them  to 
cooperate.  Even  when  the  contractor  carefully  discusses  relia¬ 
bility  and  maintainability  requirements  with  every  significant 
supplier  prior  to  assembling  his  proposal,  there  have  been 

many  serious  problems  after  award  when  the  supplier  says  "I  didn't 
think  you  really  meant  it!"  or  "Oh,  is  that  what  you  meant!" 

The  Project  Engineer  may  develop  confidence  in  the  contractors 
selection  of  suppliers  by  specifying: 

"The  contractor  shall  provide  evidence  of  the  competence  and 
willingness  of  his  proposed  suppliers  of  critical  components 
to  conduct  the  tasks  he  prescribes.  Copies  of  supplier  letters 
acknowledging  the  reliability  and  maintainability  requirements 
and  expressing  management  willingness  to  perform  the  necessary 
tasks  will  suffice." 

3.6  SHIPBUILDING  PROCUREMENTS 

A  complex  specification  situation  occurs  in  a  major  procurement 
such  as  a  class  of  ships.  The  detail  specification  for  the  ship 
class  provides  the  requirement  for  equipment  with  environmental 
conditions,  operating  conditions  and  installation  conditions. 

Weight,  space,  and  size  limitations  on  the  various  equipments 
may  be  defined.  Systems  to  be  installed  in  the  ship  may  be  procured 
by  the  government  or  the  shipbuilder.  Those  systems  purchased 
by  the  government  are,  in  general,  systems  that  are  used  in 
many  applications.  The  selection  of  specification  clauses  and 
requirements  must  be  reviewed  to  assure  that: 

(a)  The  environment  specified  in  the  equipment  specification 
is  compatible  with  its  planned  use  in  the  ship; 

(b)  The  reliability  specified  is  compatible  with  the  opera¬ 
ting  requirements; 

(c)  The  maintenance  philosophy  (types  and  numbers  of  skilled 
personnel,  spares  allowance,  level  oi  repair)  is  compatible 
with  the  situation  outlined  in  the  ship  speci f ication. 

The  systems  procured  by  the  shipbuilder  must  also  be  considered 
as  having  an  impact  on  the  reliability  of  government  furnished 
systems  and  on  the  effectiveness  of  the  ship.  The  quantitative 
requirements,  level  of  reliability  control,  and  means  of  assess¬ 
ing  the  achieved  reliability-  and  maintainability  must  be  speci¬ 
fied  for  these  systems  furnished  by  the  shipbuilder.  Quality 
control  provisions,  although  they  have  1 ittie  impact  on  the  de- 
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sign  reliability,  must  be  specified  to  assure  that  the  operational 
reliability  is  not  degraded  be].-*  an  acceptable  level. 


4  •  PROPOSAL  EVALUATION 

After  the  reliability  and  maintainability  portions  of  the  RFP 
have  been  prepared  using  the  appropriate  language  as  discussed 
above,  it  is  issued  to  the  selected  contractors.  Their  questions 
are  consolidated  and  handled  in  a  bidders  conference  that  speci¬ 
fically  invites  contractors  reliability  and  maintainability  per¬ 
sonnel.  Then  the  Project  Engineer  refines  the  detailed  proposal 
evaluation  criteria  and  organizes  to  handle  the  reliability  and 
maintainability  sections  of  the  proposals  when  they  come  in. 

Proposal  evaluation,  it  goes  without  saying,  is  a  very  complex 
and  very  difficult  subject.  Evaluation  techniques  run  all  the 
way  from  sincere  but  subjective  judgment  of  a  perhaps  unwittingly 
biased  evaluator,  to  very  complex  numerical  systems  that  generate 
precious  little  understanding  and  confidence  in  their  result. 

But  we  must  take  into  account  quantitative  contributions  to  the 
extent  feasible. 


Now  what,  precisely,  constitutes  the  "best"  proposal?  It  would 
be  easy  to  say  it  is  the  one  that  will  give  us  the  best  cost- 
effectiveness,  and  we'd  be  dead  right.  But  we  are  not  quite 
ready  to  use  the  DOD  cost-effectiveness  crite  ia,  so  it  is  dis¬ 
cussed  separately  in  section  5.3  et  al.  Let  us  discuss  an 
approacn  which  will  help  compare  contractors  apparent  adequacy 
of  proposed  programs  for  what  they  say  will  be  achieved. 

4.1 _ BROAD  EVALUATION 

First  we  will  have  a  look  at  the  broad  aspects  of  the  contractors 
proposal,  quantifying  wherever  feasible,  after  which  we  will 
evaluate  his  detailed  tasks. 


4.1.1  Cost-Effectiveness  Analysis:  For  such  provisions, 
useful  at  a  future  date,  see  section  5.4.1. 

4.1.2  Program  Requirements:  As  discussed  in  section  3.1,  it 
is  imperative  that  the  contractor  thoroughly  understand  the  re- 
("O  rements .  These  evaluation  questions  may  be  used: 
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Weight 

Rating 

Are  the  stated  environments  and  stresses 
fully  responsive? 

2  0 

Are  the  proposed  modifications  thereof 
des  irable? 

20 

Are  the  environments  and  stresses  adequately 
considered  in  design  and  analysis  approaches? 

40 

Are  the  stated  reliability  and  maintain¬ 
ability  requirements  fully  responsive? 

40 

Are  the  proposed  modifications  thereof 
des irable? 

20 

Are  the  stated  applicable  documents,  in¬ 
cluding  MIL  STD  785,  and  sections  thereof 
fully  responsive? 

40 

Are  the  proposed  modifications  thereof 
des irable? 

20 

Total 

200 

4.1.3  Program  Planning:  As  discussed  in  Section  2.1.3  and 
3.  on  Program  Plans,  and  Sections  3.2  on  Program  Charts,  we 
can  evaluate  the  plan  as  a  whole:  _ _ 


Are  all  necessary  tasks  listed  on  the 
charts  ? 

25 

Are  columns  provided  for  all  appropriate 
major  dv.:uuonts? 

15 

Are  the  appropriate  document  section 
numbers  referenced? 

10 

Is  there  a  single  responsibility  shown 
for  every  task? 

2  5 

Do  the  manweeks  for  each  task  seem  reason¬ 
able  and  sufficient,  and  are  funds  allo¬ 
cated  for  all  of  them? 

50 
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Weight 

Rating 

Do  the  total  manweeks  seem  reasonable,  com¬ 
pared  to  a  typical  range  of  5  to  20%  of 
engineering  manhours? 

25 

Does  the  timing  of  task  outputs  seem  reason¬ 
ably  relative  to  each  other  and  to  the  design 
and  production  schedule? 

15 

Ha*  every  listed  program  task  been  considered, 
and  adequate  reason  given  for  those  not 
planned? 

10 

Is  there  a  concise  description  that  gives  a 
good  picture  of  the  depth  planned  for  each 
task? 

50 

Is  there  a  clearly  identifiable,  tangible, 
auditable  output  for  every  task? 

50 

Is  there  a  statement  of  what  specific  groups 
will  use  the  output  of  each  task,  and  does  it 
seem  reasonable? 

25 

Total 

300 

4.1.4  Organization :  As  discussed  in  Section  3.4,  Buships  must 
taka  a  cloaa  look  at  tha  implications  of  the  contractors  existing 

and  proposed  organization: 


1.  Is  thara  an  existing,  as  well  as  proposed, 
reliability  and/or  maintainability  group? 

2.  Does  it  report  high  enough  to  attract  and 
hold  first-class  engineers? 

3.  Is  it  independent  or  part  of  the  engineering 
organization,  as  opposed  to  quality  control 
or  factory  operations? 

4.  If  there  are  any  past  or  present  defic¬ 
iencies,  will  they  be  remedied  by  the  proposedj 
organization? 
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6. 


8. 


f  Weight 

Rating 

Has  the  top  executive  or  the  top  engineering 
manager  issued  policies  clearly  requiring  en¬ 
gineering  design  use  of  the  reliability  and 
maintainability  technology?  Ai  e  thes°  sup¬ 
ported  by  adequate  department  policies  anu 
procedures  ? 

50 

Do  contractor  reliability  and  maintainability 
groups  have  the  authority  to  make  analyses 
and  recommendations  concerning  all  designs 
and  design  changes,  only  after  which  design 
decisions  can  be  approved? 

100 

Is  there  an  independent  channel  whereby 
design  decision  contrary  to  recommendat ion 
can  be  taken  up  the  hierarchy  for  reconsider¬ 
ation,  by  the  top  executive  if  necessary? 

50 

Is  the  work  flow  between  design  and  reliabil- 
ity/mamtainability  groups  reasonable  and 
suf f ici ent? 

50 

Total 

500 

4.1.5  Competence :  As  discussed  in  section  3.3,  wo  must  heavily 
weight  evidence  that  the  contractor  has  faced  similar  requirements 
before,  and  can  make  available  the  technical  and  management 
people  who  know  what  works  and  what  does  not: 


1. 


2. 

3. 


4. 


5. 


Has  the  contractor  done  much  design  to  quan¬ 
titatively  specified  reliability  and  main¬ 
tainability,  giving  specific  examples? 

50 

Has  he  done  much  design  to  "high"  reliabil- 
tty  requirements ,  giving  specific  examples ? 

100 

Has  he  done  much  design  and  production  in¬ 
volving  execution  of  specific  reliability 
Program  Plan  tasks,  giving  specific  examples? 

50 

Has  he  achieved  the  reliability  values 
predicted  initially? 

100 

Has  he  undertaken  many  reliability  and/or 
mai nta inabi 1 i ty  research  projects  whose  re¬ 
sults  were  put  ti  work,  and  will  they 
continue? 

50 
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6 . 


8. 


9. 


10. 


11. 


Are  his  people  active  in  government  and 
industry  committees  and  conferences? 

Wei gh  t 

Rating 

50 

Will  the  contractor  use  key  design  engineers 
on  this  program  who  have  had  reliability  and/ 
or  maintainability  design  experience? 

150 

Are  the  backgrounds  of  the  planned  reliabi  l  it* 

/ 

I 

1 

and  maintainability  specialists  adequate  for 
program? 

100 

! 

Has  the  contractor  performed,  or  is  he  per¬ 
forming  for  other  programs,  nearly  all  of  the 
required  tasks  to  about  the  same  depth? 

1 

1 

1 

1 

100 

Does  his  understanding  of  tasks  not  previ¬ 
ously  performed  seem  adequate? 

50 

Is  there  adequate  evidence  that  all  critical 
component  suppliers  are  competent  and 
willing  to  undertake  the  pr  posed  reliability 
and  maintainability  tasks? 

200 

Total 

1000 

4.2  TASK  EVALUATION 


Having  covered  the  broad  aspects,  we  can  turn  our  attention  to  the 
detailed  tasks  called  out  by  Figure  23-22  in  section  3.  and  the 
Task  Delineation  in  Section  3.3,  discussed  in  detail  in  chapter 
22.  All  weights  . hown  in  this  section  will  later  be  multiplied 
by  a  factor  of  up  to  15,  depend i no  upon  completeness  of  response, 
in  the  final  evaluation  summary.  Other  weiqhts  may  of  course  be 
used  instead,  but  it  should  be  kep^t  in  mind  that  emphasis  will 
vary  very  widely,  depending  upon  the  background  experience  of 
those  who  select  weights .  The  following  evaluation  questions  may 
be  used  : 

1.  Proqram  Plan  Update: 

Is  a  reasonable  update  procram  pi  >{*ased? 

Total 

2.  Education: 

a)  Is  there  an  adequate  management  reliabil¬ 
ity  and  maintainability  indoctrination 
or  our  am? 


10 

U) 

10 

a 

•  i 

_  1 
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Weight  ating 


b)  Is  there  an  adequate  design  engineering 
reliability  and  maintainability  training 
program? 

c)  Are  concise  reliability  and  maintainability 
reference  manuals  provided  to  engineers, 

as  well  as  selective  current  literature  on 
new  methods? 

Total 


Design  to  specified  Reliability  and  Maintain¬ 
ability  : 

a)  Does  the  design  indicate  depth  of  under¬ 
standing  of  the  reliability  and  maintain¬ 
ability  problem? 

b)  Will  the  design  meet  the  reliability  and 
maintainability  requirements? 

c)  Is  the  proposed  design  quantitative  relia¬ 
bility  and  maintainability  optimum  for 
best  cos t-e f f ect iveness ,  the  available 
tradeoffs  having  been  executed? 

d)  Are  maintenance,  logistics,  training,  etc. 
requirements  adequately  considered? 

e)  Will  all  design  specifications  for  cri¬ 
tical  components  and  changes  or  modifica¬ 
tions  thereto  contain  (1)  reliability  and 
maintainability  requirements,  (2)  verifi¬ 
cation  criteria,  (3)  protective  packaging 
requirements,  (4)  traceability  identifi¬ 
cation,  and  (5)  operating,  storage,  and 
trans  per tat  ion  environment ? 

f)  Are  requirements  beyond  state  of  the  art 
identified  correctly? 

q)  Are  components  of  inadequately-known 
reliability  identified  correctly? 

h)  Do  the  proposed  design  solutions  show 
adequate  knowledge  of  available  approaches? 

i)  Is  adequate  rationale  given  for  the 
planned  approaches? 
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I 


l 


♦ 


j)  Is  the  verification  method  and  criteria 
stated  for  each  case? 


Total 


4.  Apportionment: 

a)  Is  the  basis  of  apportionment  of  availabil 
ity,  reliability  and/or  maintainability 
logical  for  the  missions  required? 

b)  Is  the  apportionment  based  upon  best 
available  information  according  to  the 
reliability  group? 

c)  Is  the  apportionment  mathematically 
ccr rect  ? 

d)  Does  it  show  any  critical  problems  not 
identified  in  3.  above? 


Total 


5.  Model  and  Prediction: 

a'  Is  the  model  adequate  for  the  system  and 
missions  r ecu i red? 

b)  Does  it  account  for  human  reliabilitv  and 
maintainability,  if  they  are  involved? 

cl  Is  the  model  supported  by  a  practical  data 
update  system? 

d)  Is  there  adequate  provision  for  continuous 
update  and  monthly  predictions  and  com¬ 
parison  with  apportionments? 

e)  Is  design  action  required  as  a  result  of 
adverse  prediction? 


Total 


b.  Cost-Effectiveness  Analysis: 

a)  Is  there  provision  for  quarterly  update 
of  system  prediction? 

b)  Is  there  prevision  for  reporting  regular ly 
the  total  cost  saving  th  it  would  result 
from  2-to-l  MTBF  or  MTTR  improvement  ? 
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We  i  .rv 

Rati ng 

c) 

Is  des iqn  action  required  as  a  result  of 
analysis  indicating  improvement 
opportunit ies  ? 

Total 

1  ) 

2  0 

7  . 

Failure  Modes  &  Effects  Analysis: 

a) 

Will  an  analysis  be  conducted  on  every 
component,  or  someth ina  less? 

5 

b) 

Is  a  procedure  described  for  identifica¬ 
tion  of  causes,  modes ,  and  effects  of 
each  potential  failure? 

5 

c) 

Will  the  analysis  be  updated  quarterly? 

5 

d) 

Is  design  action  required  as  a  result  of 
analysis  indicating  improvement 
opportunit ies  ? 

Total 

5 

2  0 

8. 

Stress/Strength  Analysis: 

a) 

Are  such  analyses  planned  wherever  feasi¬ 
ble,  or  just  where  failure  a at  a  is  not 
obtainable,  or  not  at  all? 

b) 

Is  design  act  ion  required  as  .a  result  or 
anal  vs  is  showing  inadequ.it  e  reliability? 

5 

Tot  a  : 

Id 

9 . 

Human  Factors : 

' 

a 

where  there  is  a  eh:  ice,  is  there  adequate 
cons  i derat  i  'n  a  t  t  n e  or*,  ice  between  hurra 
at.  J  h  ar  .i*  are  r  rpum:  its? 

- 

b) 

w 111  human  enu i n* e r i n :  pr i no i p 1  os  be 
applied  adequately  to  iispl  iv.»  and  con¬ 
trols  for  the  operators? 

5 

! 

c) 

Will  hum  in  cm;  i  net,  r  1  tv:  pr  i  not  pies  be 
applied  ad*-  luately  t.  des  i  ;n  tor  -•  a  in- 
t  enance ? 

-a 

j 

d)  Is  des  ic?!»  act  1  -n  required  as  a  result, 
human  factors  analysis  in-.Ucat  irw  less 
t h an  opt  i m a  1.  d e s  i. a r,  ? 


t  a  I 


L 


^  1 
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10. 


Design  Review: 

a)  Will  documented  design  review  be  conducted 
on  all  new  designs,  design  changes,  and 
cases  of  unknown  or  suspect  reliability, 
or  something  less  than  this? 

b)  Will  they  be  conducted  at  both  conceptual 
and  pr ior-to-release  phases? 

c)  Are  they  scheduled  and  budgeted  so  that 
there  w'ill  not  be  difficulty  getting 
expert  participation? 

d)  Are  design  checklists  to  be  made  available 
to  design  engineers  prior  to  design,  for 
their  required  use? 

e)  Is  consideration  of  quantitative  relia¬ 
bility  and  maintainability  required  in 
every  review? 

f)  Are  recommendations  required  to  be  cv.tried 
as  Corrective  Action  log  items  until  re¬ 
solution  satisfactory  to  reliability  and 
maintainability  specialists? 

g)  Is  participation  limited  to  small,  effec¬ 
tive  teams  primarily  of  specialists  from 


other  than  the 
but  specifically 
maint  ai nabi 1 ity 
sent  at  ion  ? 


ible  design  group, 
including  reliibility 
and  design  group  repre- 


Total 


11.  Parts  Coro  rol : 

a)  Does  the  contractor  have  an  effective 
preferred  p.  :ts  control  program? 


W e i qh  t 

Rat  i nq 

10 

5 

r 

5 

5 

5 

40 

b)  does  the  parts  group  control  non-preferred 
part  s  qua ii.fi  cat  i on  and  approva 1  ? 

cl  Does  it  always  participate  in  Design 
Review  of  parts  assemblies? 

d 1  Does  it  provide  application  assistance 
(rich  as  derating)  and  data  to  design 
eng inoers ? 


£', 


[ 
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e)  Does  it  write  all  parts  specifications  to 
DOD  format? 

Weight 

Rating 

5 

f)  Does  it  control  parts  handling  procedure? 

5 

g)  Does  ic  control  traceability  identifica¬ 
tion? 

5 

Total 

40 

Summary  Reports: 

a)  Is  there  an  effective  reporting  system 
that  keeps  each  design  supervisor  advised 
of  the  predicted  quantitative  reliability 
and  maintainability  of  his  design,  com¬ 
pared  to  the  dpport ionmenc? 

5 

b)  is  there  an  effective  system  of  such  quan¬ 
titative  reporting  to  contractor  manage¬ 
ment  and  to  BuShips? 

3 

c)  Is  there  an  effective  system  to  audit  and 
report  monthly  progress  on  all  tasks  of 
the  Program  Plan? 

2 

Total 

10 

Corrective  Action  Control: 

a)  Does  the  contractor  have  a  system  to 
assign  individual  responsibility  fox 
corrective  action  on  every  prediction/' 
apportionment  discrepancy,  with  regular 
reporting  until  resolved? 

5 

b)  Does  it  apply  tc  every  design  review 
recommendat ion? 

2 

c)  Does  it  apply  to  every  production  and 
operational  failure? 

3 

Total 

10 

Change  &  Configuration  Control: 

a)  Does  the  change  control  system  require  in¬ 
variable  consideration  of  quantitative 
eftect  on  reliability  and  maintainability? 

5 
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a) 


b)  Does  it  require  such  effect  to  be  incor¬ 
porated  into  -he  model? 

Total 

15.  Supplier  Control: 

Axe  pre-award  surveys  made  of  supplier  de¬ 
sign  engineering  reliability  and  maintain¬ 
ability  capability,  and  the  design  and 
analysis  techniques  employed? 

Do  the  pre-award  surveys  evaluate  supplier 
management  understanding  and  support,  and 
existing  conduct  of  necessary  tasks? 

Do  the  pre-award  surveys  evaluate  achieved 
quantitative  MTSF  and  MTTR? 


b) 


c) 


d)  Axe  cost-e f fectiveness  analyses  (relia¬ 
bility  vs.  cost)  conducted  prior  to  or 
during  supplier  negotiations? 

e)  Does  the  contractor  include  quantitative 
reliability  and  maintainability  require¬ 
ments  in  every  procurement  of  a  critical 
component,  as  well  as  the  verification 
criteria? 

f)  Are  specifications,  including  environments 
and  maintenance  requirements,  issued  for 
all  critical  component  procurement,  as 
opposed  to  order  by  catalog  number? 

g)  Are  monthly  predictions  of  reliability 
•and  maintainability  required  from  every 
supplier  on  critical  components? 

h)  Are  monthly  progress  reports  required  on 
each  supplier  Program  Plan  task? 

i)  Are  resurveys  regularly  scheduled  on  all 
suppliers,  and  current  reliability  and 
maintainability  ratings  (not  quality 
ratings)  maintained? 

j)  Is  supplier  design  action  required  for 
adverse  predictions  and  verifications? 

Total 


Weight 

Rating 

5 

1C 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

50 

* 
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Weight 

Rating  j 

16.  Manufacturing  R  &  M  Control: 

i 

a)  Is  there  adequate  control  permitting  only 
known-reliability  parts  to  get  into 

i 

assemblies? 

) 

3 

b;  Is  there  adequate  handling  control?  ; 

2 

c)  Is  there  adequate  traceability  by  serial 
or  lot  number? 

3 

d)  Is  there  an  adequate  log  of  every  test? 

2 

Total 

10 

i 

1?  >  Failure  Diagnosis: 

a)  Is  there  mandatory  provision  for  every 
component  failure  to  be  recorded  through¬ 
out  design  evaluation,  production,  test, 
checkout,  and  operational  use? 

5 

b)  Is  every  such  failure  diagnosed  for  cause, 
mode,  and  effect,  with  adequate  facilities 
available? 

c 

1 

c)  Is  design  action  always  required,  wherever 
it  can  help  to  prevent  recurrences? 

!  5 

d)  Will  the  contractor  provide  diagnoses 
after  delivery,  in  operational  use? 

5 

Total 

20 

18 „  Data  Acquisition  and  Reduction: 

a)  Is  there  an  adequate  continuous  data 
acquisition  system  utilizing  data  from 
suppliers,  design,  manufacture,  trained 
field  collectors,  industry,  and  particu¬ 
larly  the  Davy  (GFE  and  operational  use)? 

5 

b)  Is  there  an  adequate  continuous  data  re¬ 
duction  system,  with  component  reporting 
in  convenient  form  for  design  engineers 
day-to-day  use? 

i 

5 

Total 

10 

t 
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Weight 

Rating 

19.  Verification: 

Is  a  complete  list  provided  of  components 
to  be  verified,  and  does  it  seem  reason¬ 
able? 

10 

b) 

Are  the  tests  adequately  specified,  pro¬ 
perly  accounting  for  environment,  and  are 
they  reasonable,  practical  and  economical, 
using  available  facilities? 

15 

c) 

Are  the  verifications  by  other  than  test 
the  best  practicably  achievable? 

5 

d) 

Is  there  an  integrated  test  plan  to  get 
the  most  information  from  the  least  number 
of  tests,  and  is  it  related  adequately  to 
the  design  and  production  schedule? 

10 

e) 

Is  design  action  required  whenever  veri¬ 
fication  is  not  obtained? 

10 

Total 

50 

4.3 

PROGRAM  PLAN  RATING 

Fully  recognizing  the  wide  variability  that  may  be  encountered 
both  in  establishing  weights  and  in  entering  an  evaluation  figure 
for  ea>.h  question,  the  errors  of  the  procedure  tend  to  balance 
each  other  out.  With  eyeball  attention  to  over  100  questions, 
and  there  are  this  many  things  deserv:'  g  of  attention,  such  a 
rating  system  will  quite  consistently  separate  the  good  from  the 
bad.  Of  course  we  should  not  expect  to  depend  upon  it  as  the  sole 
evaluation  of  the  Program  Plan,  particularly  when  total  ratings 
are  within  10%  of  each  other,  but  it  does  provide  a  powerful 
basis  for  good  human  judgment. 

Figure  23-44  is  a  chart  that  mav  be  executed  independently  by 
each  evaluator  for  each  contractors  proposal,  followed  by  dis¬ 
cussion  and  averaging  for  a  final  chart  on  each  contractor.  As 
the  evaluator  reads  each  task  of  the  proposed  program  plan,  he 
asks  himself  each  of  the  specific  questions  listed  in  section  4.2 
above.  Then  he  enters  a  summation  figure  for  each,  and  records 
it  as  Y  in  the  Specific  Content  Rating  column  of  Figure  23-44. 

As  an  example,  28  is  entered  for  "0"  Supplier  Control. 

Then  he  checks  back  for  completeness  of  that  task  statement, 
asking  himself  the  questions  in  section  3.3,  and  entering  an 
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evaluation  for  each.  The  example  for  Supplier  Control  adds  up  to 
11  which,  multiplied  by  28,  gives  308  for  the  Composite  bating. 
Manweeks  are  entered  from  the  Chart  (section  3.2)  to  get  a  feel 
for  relative  effort.  This  is  done  for  all  tasks,  to  get  a  Task 
Total,  .377  2  in  the  example. 

Similarly  the  questions  in  section  4.1  are  asked,  the  results 
entered  below  the  Task  Total,  for  a  grand  total.  Dividing  the 
example  total  of  5692  by  100,  the  Program  Plan  Rating  is  57%. 

Let’s  be  careful  about  the  meaning  of  this  rating.  It  does  not 
mean  the  evaluator  thinks  the  contractor  will  do  only  57%  of  what 
he  ought  to  do.  It  does  mean  that  he  thinks  the  contractor  plans 
to  do  about  57%  of  all  possible  things  that  could  be  done.  If 
the  contractor  can  justify  not  doing  them,  and  still  convince 
BuShips  that  he  will  more  than  meet  requirements,  so  much  the 
bettor . 

But  at  the  same  time  the  rating  is  a  measure  of  the  strength  of 
the  reliability/maintainability  program  proposed.  And  the  higher 
the  Availability  or  Reliability  proposed,  relative  to  the  state 
of  the  art,  the  higher  the  rating  should  be.  This  of  course 
suggests  taking  a  ratio  of  rating  to  Availability,  as  a  measure 
of  confidence  that  the  conti actor  can  indeed  achieve  the  proposed 
values.  Such  a  "Confidence  Rating"  is  used  below. 

4.4  EVALUATION  SUMMARY  MATRIX 


Now  that  we  have  organized  the  result  of  each  contractors  pro¬ 
posals,  and  evaluated  the  results  of  each  on  a  consistent  basis, 
we  must  evaluate  them  relative  to  each  other.  Figure  23-46  shows 
a  matrix  that  can  be  used.  Across  the  top  we  provide 

columns  for  each  contractors  response  to  BuShips  stated  require¬ 
ments  ,  and  then  another  set  of  columns  for  contractor-recommended 
alternatives.  Down  the  left  side  we  call  for  the  Confidence 
Rating  discussed  above,  and  below  that  its  key  constituents,  for 
easy  reference. 

Now  we  look  at  the  Confidence  Rating,  which  compares  Program  Plan 
Rating  Z  to  the  expected  Availability  A.  Contractor  "A”  proposed 
a  program  to  get  subs tantially  higher  Availability  of  80%  vs. 
required  70%,  by  taking  5  months  longer,  with  something  less  than 
the  anticipated  Program  Plan  strength,  so  the  Confidence  Rating 
dropped  to  71%.  Contractor  "B"  proposed  unheard-cf  Availability 
of  90%,  taking  10  months  to  get  it,  but  imposing  a  very  strong 
70%  Program  Plan,  so  the  Confidence  Rating  is  78%. 
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Obviously  these  Confidence  Ratings  are  very  rough,  only  for  use 
as  guides.  They  are  not  statistical  confidence  probabilities  of 
getting  the  stated  values,  and  they  can  go  over  100%.  While  more 
complex  methods  of  relating  Rating  to  Availability  cou^d  oe  used, 
such  refinement  does  not  seem  justified. 

4.5  DE-BRIEFING 


Nothing  is  quite  as  demoralizing  to  a  contractors  proposal  leader 
and  his  team,  who  have  put  long  hours  and  weekends  into  what  they 
believe  is  an  excellent  proposal,  than  to  be  unable  to  find  out 
why  they  lost  the  award.  Unless  they  can  find  out  what  to  do 
differently  on  the  next  one,  they  will  have  progressively  less 
interest  in  bidding.  And  such  attrition  of  competent  bidders  is 
not  in  BuShips  long-term  interests. 

The  debriefing  conference  should  specifically  invite  a  reliabil¬ 
ity/maintainability  representative  from  each  contractor,  because 
experience  has  show'n  that  others  often  do  not  understand  the 
significance  of  what  is  said  in  this  area,  and  tail  to  get  the 
information  needed.  While  the  Bureau  might  say  "That  is  the 
Contractors ' s  problem,  if  he  sent  the  wrong  people,1'  that  fact 
is  no  help  to  reliability  and  maintainability  needed  in  future 
proposals . 

while  specific  deficiencies  of  any  one  identified  contractor 
should  of  course  not  be  discussed  in  front  of  others,  that  con¬ 
tractor  does  have  the  right  to  detailed  private  discussion  of  such 
deficiencies.  The  Bureau  need  not  defend  its  decision,  but  can 
explain  just  what  was  deemed  deficient  in  relation  to  the  success¬ 
ful  proposal. 

There  would  appear  to  be  no  objection  to  giving  a  contractor  his 
own  ratings  and  average  ratings  for,  say,  the  top  three  bidders. 
With  these  his  management  can  forthwith  get  real  motivated  to 
take  action  on  future  reliability  and  maintainability  programs, 

5.  COST-EFFECTIVENESS  PROVISIONS 


While  the  cost-effectiveness  criteria  are  being  inexorably  in¬ 
serted  by  DOD,  and  they  form  the  very  basis  for  nearly  all  relia¬ 
bility  and  maintainability  decisions,  they  are  not  yet  widely 
used.  And  they  are  very  rarely  used  for  hardware  design  decisions. 


So  that  the  above  sections  will  be  closer  to  the  real  world,  we 
have  separated  out  the  cost-effectiveness  provisions  in  this 
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section.  The  numbers  in  parentheses  are  the  above  sections  of 
which  they  would  be  a  part.  The  Project  Engineer  is  urged  to  use 
them  whenever  he  sees  an  opportunity  and  advantage  in  so  doing. 

5. 1(1.2)  COST- EFFECTIVENESS  ANALYSIS 

Let's  take  a  look  at  what  DOD  3200.9  says  (1)  about  cost-effec¬ 
tiveness  in  Figure  23-49.  The  principles  apply  equally  well  at 
all  levels  of  the  system,  from  the  entire  weapon  system  thru 
ships,  down  to  parts. 

Specification  of  reliability  and  maintainability  requirements 
should  never  be  permitted  without  a  prior  analysis  of  their  true 
effect  on  system  effectiveness  affid  their  consequent  total  cost, 
as  discussed  in  chapter  26.  As  Rear  Admiral  Emerson  Fawkes  has 
said  ( 10) : 

"The  cost  of  acquisition  is  like  an  iceberg;  it  fails  to 
reveal  the  ownership  cost  of  4  to  10  times  the  acquisition 
cost . " 

Figure  23-50  shows  the  basic  tradeoffs  involved  in  using  cost- 
effectiveness  analysis  to  determine  reliability  and  maintainabil 
ity  requirements .  Such  analysis  results  may  not  always  be  used 
directly,  but  they  always  provide  remarkably  better  insight  upon 
which  to  base  good  management  judgment.  The  Project  Engineer  may 
therefore  wish  to: 

"Conduct  a  cost-effectiveness  analysis  of  the  system  for  the 
stated  missions  and  anticipated  lifetime  environments  and 
stresses,  to  determine  the  appropriate  values  of  reliability 
and  maintainability  to  be  required  of  the  contractors." 

5.1.1  (2.1.2)  Cost-Effectiveness  Analysis;  With  analyses  of 
system  effectiveness  vs.  total  long-term  cost  from  several  com¬ 
petent  bidders,  BuShips  will  be  able  to  (a)  verity  or  correct  the 
reliability  and  maintainability  quantitative  requirements,  and  (b) 
better  evaluate  bidder  competence.  See  chapter  26,  section  7.2. 

But  it  is  necessary  for  the  Bureau  to  provide  in  the  RFP  the 
criteria  for  effi  ctiveness  and  the  necessary  ownership  cost  factors, 
to  put  all  bidders  op  a  common  basis  except  for  their  own  estimates 
of  acquisition  cost.  If  such  analysis  is  requested,  as  strongly 
recommended,  the  following  RFP  words  may  be  used  by  the  Project 
"Engineer : 


"System  effectiveness  may  be  taken  as  the  product  of  (a)  per¬ 
formance  capability  measured  by  (state  empirical  formula) , 
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(b)  delivery  effectiveness  (state  formula  showing  degradation 
vs.  weeks  delay),  (c)  operational  availability  (state  formula 
combining  demand  and/or  continuous  availability  and/or  relia¬ 
bility)  and  (d)  operational  utilization  (state  formula  showing 
weight  for  each  kind  of  mission).  The  ratio  of  this  product  to 
total  cost  of  acquisition  (development  and  production)  and 
ownership  is  to  be  maximized  subject  to  (state  any  cost  or 
other  constraints).'' 

"Ownership  cost  may  be  taken  as  the  total  operational  cost 
over  (state  useful  life)  years  at  (state  hours)  actual  opera¬ 
tion  per  year,  including  (a)  operational  cost  of  (st->te  total 
$  per  manhour  for  operator  training.  Salary,  facilities,  etc.), 
(b)  maintenance  cost  of  (state  total  S/manhour  for  maintenance 
personnel  training,  salary  facilities)  plus  the  cost  of  spare 
components  plus  logistic  at  (state  $  per  pound  for  average 
components),  and  (c)  consequence  cost  external  to  the  system 
resulting  from  late  delivery  at  (state  $  cost,  if  any,  per 
week  of  delay'  and  system  failure  at  (state  $  per  hour  down, 
including  waste,  damage,  or  loss  of  equipment,  personnel,  or 
other  resources)." 

"The  contractor  snail  conduct  an  analysis  to  determine  the 
reliability  and  maintainability  that  will  achieve  maximum 
system  cost-e  f  feet  iveru  ,  over  a  range  of  r  'liability  a  tv* 
maintainability  on  each  side  of  the  state  requirements,  explain¬ 
ing  the  method  and  summarizing  res  alts  i  ,  the  Program  Plan, 
with  a  summary  of  the  design  and  manufacturing  differences  by 
which  such  ranges  would  be  obtained.  Such  MTBF  (if  used)  and 
MTTR  ranges  should  be  3-to-l  on  each  side  of  stated  require¬ 
ments.  If  the  analysis  indicates  alternative  requirements  to 
be  more  cost-effective,  he  will  recommend  new  values  of  relia¬ 
bility  and  maintainability,  but  base  the  entire  Frogram  Plan 
on  t-hs  stated  requirements." 

"Based  on  the  above  analysis,  the  contractor  will  state  what 
quant  it  at i vo  cost-e  f  f e -  t iveness  (a)  will  be  achieved  if  he 
exactly  meets  BuShips  stated  requirements,  and  (b)  would  be 
achieved  if  his  alternative  r<  -ommendations  are  accepted. 

These  estimates  and  the  means  by  which  they  are  derived,  will 
be  import  ant  cons ider at  ions  in  award  of  the  contra  . " 

5.1.2  (2.4.4)  Contracting  for  Cost-Ef feet iveness :  There  is  no 
doubt  in  the  minds  of  many  people,  including  Dr.  Hitch,  DOD  Comp¬ 
troller:  (11/  that  the  only  ultimate  solution  is  to  contract  for 
the  beit  rombinat ion  of  cost,  effectiveness,  and  delivery.  And 
he  agrees  that  delivery  can  usually  be  accounted  for  within  cost 
and  effectiveness.  But  we  do  not  yet  have  procedures  to  do  this. 
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What  we  do  not  want  is  the  lowest  proposed  price  (acquisition 
cost),  because  this  will  usually  reflect  the  Lowest  (maybe  zero) 
effort  on  reliability,  consequently  the  poorest  reliability  and 
the  highest  long-term  ownership  cost,  far  offsetting  the  immediate 
saving.  Figure  23-53  shows  the  result  of  a  Navy  survey  (12)  of 
2u  questions  to  196  contractors  in  1960,  asking  them  to  rank  what 
they  considered  to  be  the  deterrents  tc  reliability.  A  similar 
Air  Force  survey  produced  the  same  result  regarding  funding. 

Nor  do  we  want  "100%  reliability,"  because  it  would  cost  more 
than  it's  worth,  and  besides  it's  impossible.  What  we  do  w;ant  is 
the  most  system  effectiveness  (performance,  reliability  or  avail¬ 
ability,  and  delivery  adherence)  in  relation  to  total  cost  (con¬ 
tractors  price  plus  lifetime  ownership  cost  for  delivery,  delays, 
operation,  maintenance,  and  failure  consequence) .  Contracting 
for  such  a  thing  hasn't  been  done  yet,  but  it's  got  to  be  done. 

What  can  we  do? 

1.  Carefully  specify  the  taruet  reliability  and  maintainability 
values,  and  how  they  will  be  measured.  Then  require  the 
bidders  to  provide  quotations  for  a  range  of  reliability  and 
maintainability  surrounding  the  target  values  (See  1.4, 

2.1.1) 

2.  Require  the  bidders  to  conduct  their  own  analyses  of  the  cost- 
effectiveness  of  what  they  propose  to  deliver-  (2.1.2) 

3.  Award  to  the  bidder  proposing  the  best  cost-effectiveness, 
from  among  those  capable  and  whose  analyses  are  convincing. 
(4.4) 

4.  Negotiate  fixed-pr ice-incenti ve  contracts  where  the  incentive 
is  a  fraction  of  achieved  cost-effectiveness  beyond  target , 
with  coi. /erse  penalty.  In  tLvs  way  the  contractor  can  get 
paid  for  reliability  improvement  to  the  extent  that  it  pays 
off,  but  not  Otherwise- 

Value  Engineering  clauses  (13,14),  incidentally,  have  provided 
incentive  for  reduced  Acquisition  Cost,  but  not  for  improved  cost- 
effectiveness.  It  is  almost  common  for  such  proposals  to  be  dis¬ 
approved  because  the  reduced  Acquistion  Cost  is  more  than  offset 
by  consequently  higher  Ownership  Cost.  While  somewhat  higher 
reliability  usually  results  from  the  simplification,  sometimes  it 
does  not,  and  the  logistic  consequence  may  be  substantial  in 
either  case. 


DETERRENTS  TO  RELIABILITY 

US  Navy  BIMRAB  Survey 
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5.2  (4.0)  PROPOSAL  EVALUATION 


In  section  4.0  we  said  the  "best"  proposal  is  undoubtedly  the  one 
that  will  give  us  the  best  cost-effectiveness.  But  even  if  we 
have  cost-effectiveness  analyses  from  several  contractors  snowing 
what  cost-effectiveness  they  will  achieve,  how  do  we  knew  they 
can  do  it?  First  let's  have  a  look  at  the  DOD  directive.  In 
Figure  23-55  DOD  says  the  basis  for  decision  shall  never  be  cost 
alone,  and  Figure  23-49  says  the  idea  is  to  get  an  "optimum  bal¬ 
ance  between  total  cost,  schedule,  and  operational  effectiveness." 

This  is  the  same  as  maximum  cost-effectiveness  if  schedule  slip¬ 
page  effects  are  included  in  cost-effectiveness. 

With  this  objective  in  mind,  let's  discuss  some  approaches  which 
will  help  compare  contractors  for  (a)  overall  program  cost-effec-  1 

tiveness,  and  (b)  reliability/maintainability  program  cost-effec¬ 
tiveness  . 

5.3  (4.1)  BROAD  EVALUATION 

5.3.1  (4.1.1)  Cost-Effectiveness  Analysis:  If  BuShips  h.-vs  asked 
the  contractors  for  their  own  cost-e f fectiveness  analyses,  then 
there  are  three  things  to  evaluate.  They  are  [a)  the  quality  of 
the  analysis,  (b)  the  actual  program  cost-effectiveness  figure 
they  propose  to  achieve,  and  (c)  the  specific  reliability/maintain¬ 
ability  program  cost-effectiveness  to  the  extent  we  can  isolate  it. 

For  quality  of  the  analysis  these  questions  may  be  used: 


Weight 

Rating 

1. 

Does  the  analysis  show  correct  interpre¬ 
tation  of  requirements? 

100 

2. 

Does  it  show  understanding  of  the  problem? 

100 

3. 

Is  the  analysis  understandable  and  logical? 

100 

4. 

Is  it  based  on  good  data,  or  where  not,  is 
it  shown  that  probable  data  range  will  not 
affect  conclusions? 

100 

5. 

Are  the  conclusions  logical  and  correct? 

100 

Total 

500 

Turning  now  to  the  actual  program  cost-effectiveness  arithmetic. 
Figure  23-56  shows  a  simple  calculation.  Let  us  say  that  BuShips 
has  specified  a  performance  capability  that  is  normalized  to  100% 


COST-EFFECTIVENESS  ARITHMETIC 
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for  the  stated  requirement.  That  is,  90%  performance  would  mean 
that  capacity,  speed,  and/or  accuracy  have  been  somehow  comprom¬ 
ised  so  that  system  effectiveness  is  reduced  10%. 

Let  us  say  that,  the  BuShips  requirement  for  Availability  is  7  0%, 
meaning  that  experience  has  shown  this  kind  of  complex  system  to 
be  out  of  commission  30%  of  the  time.  And  let's  say  required 
Schedule  Effectiveness  is  100%,  but  in  the  RFP  we  have  advised  the 
contractors  that  Effectiveness  to  the  Navy  drops  1%  for  every  month 
of  slippage  beyond  the  specified  delivery  date.  Now  we  can  get 
required  and  expected  system  Effectiveness  as  the  product  of  these 
three  factors,  or  70%. 

Anticipated  Acquisition  costs  we'll  say  are  $22  million,  of  which 
$2  million  is  expected  on  the  reliability/maintainability  programs. 
And  based  upon  the  expected  reliability  and  maintainability  be¬ 
hind  the  70%  Availability,  we  expect  resultant  Ownership  cost  to 
be  about  $48  million  over  the  systems  useful  lifetime.  A  very 
modest  expectation.  Therefore  expected  Total  Cost  is  the  sum,  or 
$70  million.  Cost-effectiveness  is  the  ratio  of  Effectiveness  to 
Total  Cost,  or  1.0  percent  per  $  million. 

Now  let's  say  contractor  "A"s  analysis  indicated  optimum  Avail¬ 
ability  to  be  80%,  and  the  extra  work  to  get  it  would  slip  sched¬ 
ule  by  5  months.  Put  the  result  would  be  considerably  better 
Effectiveness  of  76%.  Moreover  the  consequent  drop  of  Ownership 
Cost  results  in  a  Cost-Effectiveness  improvement  of  40%  over  the 
requirements.  However  it  would  cost  an  extra  $1  million  for 
Acquisition  to  get  that  $15  million  saving. 

Contractor  "B"s  analysis  led  him  to  recommend  getting  90%  Avail¬ 
ability,  partly  by  some  sacrifice  of  Performance  Capability  to 
95%,  and  by  taking  10  months  longer  for  a  special  stress/strength 
test  program.  This  nets  about  th^  same  Effectiveness,  77%,  but 
remarkable  reduction  of  Ownership  Cost.  The  result  is  a  90% 
improvement  over  anticipated  Cost-Effectiveness. 

5.4  (4.4)  EVALUATION  SUMMARY  MATRIX 

When  the  contractor  has  been  asked  to  state  the  cost-effectiveness 
he  will  achieve,  a  much  more  powerful  evaluation  is  feasible. 

Figure  23-46  can  be  expanded  as  in  Figure  23-58. 

Down  the  left  side  we  provide  for  entries  of  (a)  total  program 
Cost-Effectiveness,  because  afl~r  all  this  is  the  true  and  over¬ 
riding  objective,  (b)  the  reliability/maintainability  program 
Cost-Effectiveness  to  be  explained  shortly,  and  (c)  the  Confidence 
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Rating.  Again  wp  list  some  of  the  key  constituents  of  the  figures 
above,  for  easy  reference. 

If  we  multiply  Availability  (and/or  Reliability)  A  by  the  Delivery 
Effectiveness  factor  attributable  to  Availability  achievement  Dr , 
we  have  a  number  for  the  Effectiveness  contribution  resulting  from 
the  reliability/maintainability  program.  Then  if  we  add  to  the 
contractors  reliability/maintainability  program  cost  Cr  the  resul¬ 
tant  lifetime  Ownership  Cost  directly  attributable  to  reliability/ 
maintainability  achievement  (such  as  maintenance,  logistics, 
failure  consequence,  etc.)  we  have  the  Total  Cost  to  BuShips  of 
program  reliability/maintainability  or  lack  thereof.  Dividing 
one  oy  he  other  we  get  a  Reliability/Maintainability  Program 
Cost-Effect iveness . 

For  Total  Program  Cost-Effectiveness  we  note  the  example  figures 
developed  in  section  5.4.1  above.  Reliability/Maintainability 
Program  Cost-Effectiveness  figures,  derived  from  the  same  data  as 
explained  above,  show  1.4,  2.2,  and  3.7%  per  $  million  spent. 

Note  the  greater  leverage  compared  to  the  Total  Program  figures. 

In  this  example,  if  the  70%  truly  reflects  an  excellent  and  con¬ 
vincing  Program  Plan,  BuShips  could  undoubtedly  decide  on  contrac¬ 
tor  "B's  recommended  approach.  Even  if  contractor  " B"  doesn't  do 
as  we] I  as  everybody  expects,  the  saving  of  $13  million  compared 
to  contractor  “A“s  proposal  leaves  a  lot  of  leeway  for  corrective 
action  as  problems  crop  up. 
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6.  SUMMARY 

In  this  chapter  we  have  reviewed  the  DOD  and  CNO  directives  to 
achieve  a  cost-effective  optimum  reliability  and  maintainability, 
and  the  steps  BuShips  management  and  Project  Engineers  can  take 
to  plan  and  organize  such  programs. 

We  have  discussed  the  complete  dependency  of  the  reliability  and 
maintainability  program  upon  (a)  the  basic  operational  needs  and 
(b) ,  in  the  future,  the  cost-effectiveness  balance. 

Ir.  the  proposal  phase  there  must  be  a  dialogue  between  the  Bureau 
and  contractors  until,  by  virtue  of  their  estimated  cost  to  achieve 
various  reliability  values,  the  optimum  values  can  be  determined. 
This  is  followed  by  detailed  planning  of  the  contractor  tasks 
necess  ary  . 

Reliability  and  Maintainability  Program  Plans,  like  any  contract. 


must  be  the  sole  vehicle  of  Bureau/contractor  reliability  and 
maintainability  understanding,  and  be  kept  that  way  as  design 
progresses . 


Tasks  rnuot  be  very  explicitly  stated,  particularly  describing 
exactly  what  output  will  be  produced,  who  needs  it,  and  who  does 
it.  Anythin  i  ’e«s  produces  no  worthwhile  result. 

Proposal  evaluation  is  developed  quantitatively,  to  facilitate 
fair  comparison  of  contractors  proposed  plans.  But  in  view  of 
the  DOD  directives  to  use  cost-effectiveness  criteria,  suggested 
approaches  are  provided.  We  have  modified  what  has  worked  a1! 
with  a  point  of  departure  that,  if  we  try,  ought  to  work.  T..ert 
is  no  doubt  about  the  soundness  of  the  DOD  principles,  but  imple¬ 
mentation  will  take  much  patience. 

Program  Direction  can  be  nothing  but  fire  drills  if  the  Prugra 
planning  is  incomplete  or  inadequate.  With  a  solid  Projram  Pi 
and  real  understanding  between  BuShips  and  the  contractoi ,  any 
good  contractor  will  make  BuShips  reliability  and  maintainability 
Program  Direction  no  problem. 
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Chapter  24 
PROGRAM  MANAGEMENT 

This  chapter  continues  with  the  broad  content  of  reliability  and 
nvaint  a  inability  programs,  with  emphasis  upon  the  Bu'Ships  manage¬ 
ment  actions  necessary  to  conduct,  evaluate,  and  control  such 
programs  to  match  the  requirements.  Chapter  23  covered  the 
planning  and  organizing  of  such  programs. 

Unless  someone  has  the  clear  responsibility  for  reliability  and 
maintainability  management  and  technical  decisions  throughout  a 
program,  the  required  results  will  not  be  achieved.  Ideally  it 
would  be  very  desirable  to  have  a  single  individual  responsible 
for  all  reliability  and  maintainability  analysis  and  decisions 
on  any  one  program  from  "womb  to  tomb"  (say  from  Proposed 
Technical  Approaches  to  end  of  system  useful  life).  Let's  review 
the  DOD  framework  and  BuShips  responsibility  structures  for  (a) 

RDT.  E  and  (b)  shipbuilding  programs. 

1.  RESEARCH  AND  DEVELOPMENT 

Historically,  preliminary  feasibility  studies,  systems  analyses 
and  advanced  development  efforts  have  been  steps  leading  to  the 
initiation  of  full-scale  development  of  a  new  weapon  or  support 
system.  Then,  after  project  approval  based  upon  a  Technical 
Development  Plan  (TDP) ,  the  Departments  would  prepare  a  Request 
for  Proposals  (RFP) .  The  proposals  selected  from  those  submitted 
led  to  the  negotiation  of  a  contract  (almost  universally  of  the 
cost-plus -fixed- fee  type)  for  an  Engineering  Development  or 
Opt  rational  Systems  Development. 

Experience  over  a  period  of  years  has  shown  that  these  practices 
have  frequently  led  to  "brochuremanship"  in  proposals,  unrealistic 
technical  definition  of  required  system  characteristics  (including- 
reliability) ,  neglect  of  the  need  for  further  exploratory  effort 
prior  to  full-scale  development,  incomplete  planning,  and  overly 
optimistic  contractor  proposals  which,  in  turn, have  been  pursued 
by  the  cognizant  Department  of  DOD.  These  practices  have  resulted 
in  disruptive  or  untimely  performance  or  design  changes,  large  cost 
overruns  on  development  contracts  and  significant  schedule  slippages 
and  have  degraded  the  effectiveness  of  the  operational  units  and 
escalated  the  total  cost  (including  production,  operation  and 
maintenance) . 


Further,  some  projects,  which  would  never  have  been  started  if 
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total  consequences  had  been  foreseen,  have  been  canceled  after 
partial  development,  with  attendant  financial  losses.  in  addition, 
since  the  demand  for  increased  funds  for  the  project  that  was  in 
trouble  reduced  funds  available  for  other  projects,  thus  bringing 
about  their  "starvation"  and  causing  them  to  be  stretched  out  in 
time  or  canceled. 

To  overcome  these  problems,  a  number  of  new  management  tools  have 
been  used  in  the  last  several  years  —  tools  net  necessarily  new 
in  concept  but  new  in  emphasis.  Among  these  are  thi  current  OSD 
Programming  System,  incentive  contracting,  PERT  and  PERT/Cost, 

Value  Engineering,  contractor  performance  evaluation,  cot;t-ef  fect- 
iveness  analysis,  categorization  of  research  and  development  (R&D) 
and  the  Project  Definition  Phase  (PDP) . 

Following  the  encouraging  initial  applications  of  PDP,  the  Director 
of  Defense  Research  and  Engineering  (DDR&E)  directed  the  application 
of  PDP  to  major  development  projects  meeting  certain  criteria.  DOD 
Directive  3200.9  expanded  the  application  of  PDP  and  provided 
fundamental  direction  for  the  conduct  of  PDP. 


1.1  CONCEPTUAL  PHASE  0 

While  the  application  of  PDP  is  not  our  primary  concern  here,  lack 
of  reliability  had  much  to  do  with  bringing  it  about.  And  the 
underlying  principles  of  PDP  can  and  should  be  used  for  all  R&D 
programs.  Let  us  trace,  in  Figure  24-4,  the  path  of  an  R&D  pro¬ 
ject  from  inception. 

The  Chief  of  Naval  Operations  (CNO) ,  cognizant  of  a  need,  frames 
and  issues  a  Naval  Research  Requirement  (NRR) ,  an  Exploratory 
Development  Requirement  (EDR) ,  or  a  General  Operational  Require¬ 
ment  (GOR) .  Exploratory  development  progresses,  and  is  reported 
or  a  DD613  form  to  the  CNO  and  to  the  Director  of  Defense  Research 
and  Engineering  (DDR&E) . 

When  CNO  has  a  specific  operational  problem  that  requires  develop¬ 
ment,  it  prepares  a  Tentative  Specific  Operational  Requireir.ent 
(TSOR) .  It  is  at  this  point  that  the  tentative  reliability, 
maintainability  and/or  availability  requirements  must  be  expressed, 
for  they  will  have  an  impact  on  the  consideration,  selection,  and 
treatment  of  Proposed  Technical  Approaches  (PTA) .  The  PTAs,  in 
tur P-  FRuifc  express  what  reliability,  maintainability  and  avail¬ 
ably  cy  is  achievable  by  each  approach. 


w 
o 
£ 
w 

g* 

W  H 

w  3 


o 

CO 

o' 

o 

<n 

co 


w 

co 

< 


> 

z 

pm 

o 


24-5 


Independently ,  NRRs  or  EDRs  cause  Exploratory  Development,  result¬ 
ing  in  PTAs  which  should  include  approaches  to  achieve  reliability 
and  maintainability.  In  this  case  the  result  is  Advance  Develop¬ 
ment  Objectives  (ADO),  which  should  contain  reliability  and  main¬ 
tainability  needs  in  the  stated  quantitative  objectives. 

Consideration  of  the  PTAs  leads  to  a  firm  Specific  Operational 
Requirement  (SOR) ,  which  must  include  reliability  and  maintain¬ 
ability  figures.  From  this  BuShips  prepares  a  Program  Change 
Proposal  (PCF)  for  Office  of  the  Secretary  of  Defense  (OSD) 
approval  in  relation  to  Five  Year  Force  Structure  and  Financial 
Program,  and  a  preliminary  Technical  Development  Plan  (TDP) . 

SECNAV  3900.14a  on  the  Reporting  of  RD&E  Program  Information 
transmits  DOD  Instruction  3200.6  of  7  June  1962  on  the  same  sub¬ 
ject  (2).  Sections  IIIB2a  of  3200.6  states  that  the  TDP  will 
include: 

"A  narrative  statement  of  the  requirements,  a  brief  develop¬ 
ment  plan,  and  statements  delineating  the  performance, 
reliability  and  maintainability  characteristics.  (See 
Inclosure  2  for  applicable  categories  and  types  of  relia¬ 
bility  and  maintainability  information  to  be  included.)" 

Inclosure  2  of  3200.6,  sections  A  and  C,  state: 

"TDPs  will  normally  include  the  kinds  of  information  (listed 
below) .  Comprehensive  reliability  and  maintainability  pro¬ 
grams  for  feasibility  studies,  exploratory  development  and 
Advanced  Development  categories  are  not  desired.  However, 
due  consideration  shall  be  given  tc  all  characteristics, 
including  reliability  and  maintainability,  in  the  early 
planning  and  feasibility  study  stages,  and  comprehensive 
reliability  and  maintainability  programs  are  expected  for 
operational  development  projects.  It  is  intended  that  both 
the  human  and  hardware  aspects  of  reliability  and  maintain 
be  considered.  The  goal  is  a  balanced  and  integrated  effort 
aimed  at  optimizing  operational  effectiveness,  total  cost 
and  early  availability." 

"Normally  (Research  and  Exploratory  Development),  which  are 
essentially  the  requirements,  will  be  precisely  and 
quantitatively  stated.  Information  in  the  TDP  responsive 
to  Advanced  Developments  and  Engineering  Developments  should 
outline  the  plans  for  achieving  reliability  and  maintainability, 
including  the  significant  elements.  In  some  cases,  listing  of 
a  significant  element  without  detail  is  satisfactory.  For 
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example,  indication  that  reliability  apportionment  and 
prediction  is  a  part  of  the  reliability  program  may  ba 
sufficient.  In  other  cases,  for  example,  reliability  test 
and  demonstration,  and  principal  details  of  the  planned  program 
are  necessary.  Whenever  there  it  heavy  emphasis  or  unusual 
treatment  of  an  element,  this  should,  of  course,  be  detailed." 

Accordingly  the  TDP  must  include  a  reliability  ai-.l  maintainability 
Program  Plan,  as  shown  in  section  3.  of  chapter  23.  The  PCP  and 
preliminary  TDP  are  routed  to  OSD  for  PCP  approval  and  go-ahead, 
which  authorizes  preparatxon  of  a  firm  TDP  (and  revised  PCP  if 
needed),  a  PDP  Plan  (if  used),  a  Request  for  Proposal  (RFP) , 
and  preparation  of  evaluation  criteria.  Content  of  the  RFP,  so 
far  as  reliability  nnd  maintainability  r re  concerned,  is  covered 
by  section  2.1  of  chapter  23. 

1.2  DEFINITION  PHASE  I 

If  PDP  is  used.  Figure  24-7  shows  the  proposals  resulting  from 
the  above  RFP,  and  proposal  evaluation  is  described  in  section 
4.  of  chapter  23.  Contract  negotiation  is  discussed  in  section 
4.  herein.  In  this  case  the  contract  results  as  in  Figure  24-8, 
in  proposals  for  Phase  II  full-scale  design,  development  and 
production,  and  these  proposals  in  turn  can  be  evaluated  as  in 
section  4.  of  chapter  23.  After  the  necessary  refinement  of 
the  TDP  (including  its  Dependability  Plan)  and  PCP,  and  dual 
negotiation,  a  final  ACQUISITION  Phase  II  contract  is  negotiated. 

If  r'h'P  is  not  used,  uhe  procedure  is  identical  except  that  the 
steps  between  the  Proposals  for  Phase  I  block  (Figure  24-7)  and 
the  Proposals  for  Phase  II  block  (Figure  24-8)  are  omitted. 

1 . 3  R&D  MANAGEMENT 

For  BuShips  management  and  execution  of  the  Research,  Development, 
Test  and  Evaluation  Program,  Figures  27-9  and  24-10  from  reference 
(1)  show  very  concisely  the  distribution  between  Program  Manager 
and  Technical  code  responsibilities.  In  concept  the  Program 
Manager  is  responsible  for  determining  what  is  to  be  done  to  meet 
Bureau  requirements,  and  the  Technical  Code  is  responsible  for 
determining  how  to  execute  the  selected  tasks  within  the  time- 
cost-performance  framework. 

Clearly  it  is  the  job  of  the  Project  Engineer  in  the  Technical 
Code  to  (a)  determine  how  to  achieve  the  required  reliability 
and  maintainability,  to  (b)  recommend  which  tasks  are  acceptable, 
(c)  determine  method  of  accomplishment,  (d)  prepare  Proposed 
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Technical  Approaches  accounting  for  reliability  and  maintain¬ 
ability,  speci  fications  and  RFPs  to  achieve  required  vrj.ues, 

(e)  administer  the  technical  contract,  and  (f)  initiate  any 
required  changes  of  specified  reliability  and  maintainability . 

The  Technical  Codes,  whetner  or  not  PDP  is  used,  are  responsible 
for  hardware  design  and  evaluate  activities,  and  responsible  for 
making  certain  that  reliability  and  maintainability  requirements 
are  invariably  considered  and  specified  in  hardware  PTAs ,  TDPs, 
and  P.FPs ,  evaluate  the  resulting  proposals  for  reliability  and 
maintainability  content,  recommend  awards,  participate  in  con¬ 
tract  negotiation  of  reliability  and  maintainability  content, 
and  administer  contractor  reliability  and  maintainability  per¬ 
formance  . 

For  example  in  preparing  PTAs,  the  Technical  Code  may  have 
evaluated  one  or  more  unsolicited  proposals  in  the  same  area, 
noting  the  reliability  and  maintainability  these  contractors 
expected  to  be  able  to  achieve.  He  must  insure  that  the  probable 
reliability  effects  of  departures  from  existing  designs  are 
evaluated,  that  likely  reliability  problem  areas  are  identified 
(e.g. ,  perhaps  microcircuitry  is  involved,  and  thermal  sensitivity 
of  the  vemi- conductors  is  known  to  be  causing  serious  problems  in 
such  circuitry;  the  PTA  should  indicate  how  this  would  affect 
system  reliability  and  what  approach  would  be  used  to  minimize 
adverse  effect). 

In  preparing  a  Technical  Development  Plan,  he  includes  a  Depend - 
abi  ity  Plan  that  is  firmly  based  on  mission  requirements  and  system 
performance  parameters.  He  must  assure  himself  that  the  plan : pro¬ 
vides  for  resolving  the  problems  of  interface  with  other  shipboard 
systems  and  that  it  identifies  any  areas  which  will  require  special 
reliability  upgrading  effort.  If  underwater  pickup  resistance  to 
corrosion  must  be  improved  by  an  order  of  magnitude,  he  must  be  o 
sure  that  the  Dependability  Plan  provides  for  such  effort,  that 
it  shows  the  type  of  testing  required  to  verify  this  new  resistance, 
and  that  cost  and  schedule  implications  are  clearly  indicated. 

He  must  also  insure  that  the  remaining  sections  of  the  TDP  reflect 
this  requirement.  The  engineering  plan  must  contain* provisions  j  ' 
for  appropriate  development,  the  schedule  must  allow  for  it,  the- 
cost  must  be  included  in  the  overall  cost  breakdown,  etc.  ; 

2.  SHIPBUILDING  PROGRAM  ^  ' 

As  detailed  in  OPNAV  4700. 12B,  the  "development  of  ship  ,  1 

characteristics  and  authoritative  cost  estimates  is  a  complex 
and  lengthy  procedure,  requiring  inputs  from  the  Fleet,  different 
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..our  res  within  Oi-'MAV ,  cognizant  technical  bureaus  and  the  Bureau 
‘ a  f  N a \  a  1  I  e r s  n n e .1 .  B a s  i  c  c u  i ;  1  a n c o  i s  p r o v  i d e d  t >v  M iss  i o n  and 
Tasks  as  approved  by  the  Chief  of  Naval  Operations .  The  in for - 
conta  inod  in  the  Mission  and  T..e,ks  is  amplified  by  single  page 
character ist ics  which  are  prepared  by  the  type  sponsor,  approved 
by  the  Deputy  Chief  of  Naval  Operations  (pi  not  Operations  and 
Readiness)  and  provided  to  the  Chairman,  Ship  Characteristics 
Board.  These  single  page  characteristics  delineate  the  signifi¬ 
cant  features  and  capabilities  which  form  the  basis  of  cost  and 
feasibility  studies  by  the  Chief,  Bureau  of  Ships,  and  the  re¬ 
sultant  development  of  the  initial  credible  price  estimate. 

Based  on  the  above  information  and  guidance,  approved  ship 
characteristics  are  developed. 


’’The  s 
s  h  i  p  p 
is  t  i  cs 
pur  pos 
the  sh 
M  is  s  io 
an  exc 
result 
act.ua  1 
could 


tatoment  of  Mission  and  Tasks  for  each  type  of  U.S.  Naval 
rov ides  the  key  to  a  ship's  ultimate  capabilities,  character- 
arid  cost.  Because  it  furnishes  a  bread  statement  of  the 
e  for  ■  hich  the  ship  is  to  be  designed  and  the  tasks  which 
ip  can  be  expected  to  accomplish,  each  word  used  in  the 


n  and  Tasks  statement  is  significant.  The  stipulation  of 
essive  capability  in  cither  the  mission  or  tasks  may  well 
in  an  overly  complex  ship  priced  at  a  cost  which  jeopardizes 
construction.  Similarly,  understatement  of  capability 
result  in  a  sh i p  of  less-than- desirable  operational  qualities." 


.  i 


MISSIONS  AND  TASKS 


Figure  24-  .1 3  shows  the  sequence  leading  up  to  transmission  of  the 
Mission  and  Task  statement  for  each  ship  to  the  Chairman  of  the 
S h i ps  C h a r a e ter i s tic  Bo a r d . 


The  reliability  and  maintainability  requirements  are  not  being 
included  in  the  recommended  Tasks,  nor  in  the  1-page  Ships  Character¬ 
istics  , today.  But  the  incorporation  of  such  requirements,  where 
applicable,  will  ut l mutely  avoi d  or  minimize  many  problems  such  as 
(a)  excessive  system  "out  of  commission”  time,  (b)  degraded  weapon 
system  effectiveness,  (c)  impractical  maintenance  manpower  and 
skill  requirements,  and  (d)  pre-emption  of  new  weapon  system  funds 
co  pay  for  the  maintenance. 


S !  I  r  P  S  C  H  AR  ACT  E  R I  ST  ICS 


Figure'  24-14  shows  the  next  sequence  leading  to  final  PCP  approval 
by  ts  '  Secretary  of  defense.  BuShips  primary  point  of  contact,  for 
ships  char  act  er  i  s  t  and  cost.,  is  with  the  Ships  Character  ist  ic 
Boa  1  (sen)  ,  but  Bv..  nips  is  not  considering  their  feasibility  or 
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SHIPS  CHARACTERISTIC  DEVELOPMENT 
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cost.  It  would  appear  that  BuShips  can  help  avoid  those  future 
fleet  and  budget  problems  by  finding  out  from  SCB  how  much  relia- 
uiliuy  mid  infill  La  in  cuj  i  li  Ly  j.»  .leeded,  and  including  its  cast  and 
its  schedule  impact. 

2.3  SHIPBUILDING  MANAGEMENT 

Referring  to  Figure  24-16,  the  Ships  Characteristics  Board  (SCB), 
in  response  to  the  Mission  and  Task  statement  issued  by  the  Chief 
of  Naval  Operations  (CNO) ,  delineates  the  desired  characteristics 
to  the  BuShips  Ships  E>esign  Division.  Although  such  desired 
characteristics  do  not  yet  include  the  reliability  and  maintain¬ 
ability  characteristics  desired,  they  should  be  included  as 
rapidly  as  feasible. 

The  Ships  Design  Division,  like  the  Program  Manager  Code  for 
RDT&E,  delineates  the  cask  of  determining  feasibility,  character¬ 
istics  and  cost,  to  be  executed  by  the  cognizant  Technical  Code. 
The  Technical  Code  may  need  some  help  from  one  or  more  contrac¬ 
tors,  for  which  contracts  are  placed.  The  resulting  report  of 
feasibility,  characteristics  and  cost  should  include  the  relia¬ 
bility  and  maintainability  characteristics  and  cost.  It  is  then 
transmitted  back  to  SCB  via  the  Ships  Design  Division. 

After  the  SCB  has  made  final  recommendations  to  the  Standing 
Committee  (Figure  24-14)  and  the  program  is  approved  by  the 
Secretary  of  Defense,  CNO  issues  authorization  to  the  BuShips  Type 
Desk  to  proceed  with  acquisition.  The  Type  Desk  integrates  the 
entire  program.  For  shipbuilding  and  conversion  it  deals  direc¬ 
tly  with  the  contractors,  including  preparation  and  issuance  of 
RFPs  which  should  include  any  reliability  and  maintainability  re¬ 
quirements.  When  contractors  proposals  are  received  the  Type 
Desk  selects  a  contractor  and  negotiates  a  contract,  which  should 
include  any  reliability  and  maintainability  requirements. 

For  GFE  equipment  the  Type  Desk,  again  like  the  Program  Manager 
Code  for  RDT&E,  authorizes  the  Technical  Code  to  prepare  and 
issue  final  P.R.  or  Requests  for  Proposal,  the  reliability  and 
maintainability  content  of  which  should  be  as  in  chapter  23. 

The  contractors  offer  their  proposals,  which  the  Technical  Code 
evaluates,  consolidates  the  best  features  of  all,  may  survey  one 
or  two  bidders,  and  makes  its  final  recommendation  to  the  Type 
Desk.  After  its  decision  the  Technical  Code  debriefs  the  losers 
and  proceeds  with  contract  negotiation.  The  contract  should  con¬ 
tain  a  reliability  and  maintainability  Program  Plan  as  detailed 
in  chapter  23. 


SHIPBUILDING  R&M  RESPONSIBILITIES 
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Throughout  performance  of  the  contract  the  Supervisor  of  Ship¬ 
building  (SupShip)  or  local  cognizant  government  representative 
audits  contractor  performance,  which  should  specifically  include 
reliability  and  maintainability  progress.  He  should  conduct 
periodic  surveys  of  contractor  reliability  and  maintainability 
capability.  He  directs  the  contractor,  and  reports  progress  to 
the  Type  Desk.  The  Type  Desk  of  course  must  approve  or  redirect 
the  Supervisors  actions  as  necessary. 

Having  discussed  the  RDT&E  and  shipbuilding  workflow,  we  can  turn 
■cur  attention  to  some  methods  found  effective  in  industry. 


3.  INTERNAL  RESPONSIBILITY  ASSIGNMENT 


While  the  above  BuShips  allocation  of  responsibilit-  says  that 
some  code  is  responsible  for  reliability  and  maintainability  at 
all  times,  that  will  not  get  the  job  done  within  the  respective 
codes.  The  clichfe  "Reliability  is  everybodvs  business"  means 
that  ’-'obody  feels  personally  responsible  to  do  much  about  it, 
except,  perhaps,  to  give  lip  service. 

So  it  is  extremely  important  that  every  code  management  issue 
and  enforce  internal  instructions  that  (a)  name  an  individual 
specifically  responsible  for  reliability  and  maintainability  on 
every  program  (one  may  handle  several  programs),  (b)  cause  the 
named  individuals  to  get  educated  on  this  subject,  and  (c)  re¬ 
quire  them  to  know  program  reliability  and  maintainability  status 
at  all  times.  We  are  not  suggesting  additional  manpower.  We 
are  suggesting  that  every  affected  Code  should  train  say  10%  of 
.Its  people  to  become  real  reliability  and  maintainability  experts. 

This  is  particularly  important  in  SupShip  and  InsMat  offices 
cognizant  over  contractors  plants,  where  he  can  get  close  to  the 
coni" actor  counterparts  and  understand  the  problems. 

4  *  CONTRACT  NEGOTIATION 

See  chapter  23  section  2.4.  Normially  the  BuShips  financial 
contract  negotiator  and  the  Project  Engineer  will  not  be  suffic¬ 
iently  expert  on  detailed  reliability  and  maintainability  pro¬ 
blems  and  impact  to  judge  consequent  contractual  content.  There¬ 
fore  it  is  imperative  that  at  least  one  BuShips  reliability/ 
maintainability  specialist  participate  in  the  reliability  and 
maintainability  Program  Plan  and  incentive  (if  used)  negotiation. 
And  it  is  just  as  imperative,  for  the  same  reason,  that  at  least 
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one  contractor  reliability/maintainability  group  supervisor 
participate . 

The  BuShips  reliability /maintainability  specialist  should  re com¬ 
ment  (a)  exact  final  wording  of  the  Requirements,  including  the 
exact  means  of  verification  of  their  achievement,  (b)  exact  con¬ 
tent  of  the  Program  Charts,  (c)  exact  content  of  all  Task  Delin¬ 
eation  and  should  (d)  participate  in  negotiation  and  approval  of 
the  proposed  reliubilicy/maintainabili ty  organization.  He  should 
(e)  recommend  and  concur  with  the  incentive  relationship  to  cost- 
effectiveness  so  far  as  reliability  and  maintainability  are  con¬ 
cerned.  He  must  not  concur  with  a  relationship  which  fails  to 
compensate  the  contractor  for  reliability/maintainability  expen¬ 
ditures  that  will  clearly  improve  cost-ef feetivcness . 


5  *  CONTRACTOR  EVALUATION 

This  section  is  concerned  with  the  evaluation  of  a  contractor's 
capability  to  achieve  required  reliability  and  maintainability, 
as  well  as  evaluation  of  his  actual  achievement  during  the  con¬ 
tract  . 

5.1  PRE-AWARD  SURVEYS 


Prior  to  contract  award  there  should  be  a  comprehensive  survey 
of  the  reliability  and  maintainability  competence  of  the  one  or 
two  contractors  still  under  consideration.  At  this  point  the 
Proposal  Evaluation  questions  of  chapter  23  can  be  u«ed  directly, 
except  for  the  few  as  outlined  in  Figure  23-18,  that  may  not  be 
applicable.  Much  more  detailed  surveys  (1)  may  be  used  if 
warranted,  but  they  are  costly  and  time-consuming. 

In  order  to  provide  uniformity  of  assessment  across  all  contrac¬ 
tors  ,  Technical  code  609  is  responsible  for  all  BuShips  surveys 
of  contractors  reliability  and  maintainability  competence.  Upor 
request,  code  609  will  execute  this  standard  sequence: 

1.  Schedule  surveys. 

2.  Assign  a  survey  chairman  (INM  or  Supships) 

3.  Notify  the  contractor  and  cognizant  INM  or  BuShips 

personnel  by  letter  at  least  30  days  prior  to  the 

proposed  survey. 

4.  Conduct  a  pre-survey  conference  of  the  team  with  con¬ 

tractor  personnel. 

5.  Conduct  the  survey  and  evaluate  the  program  against  the 

pertinent  items  of  section  5.0. 
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6.  Conduct  a  post-survey  critique  for  the  contractor  and 

obtain  contractor  response. 

7.  Prepare  a  final  report  to  the  requesting  Technical  Code, 

copies  to  team  members. 

8.  Follow  up  with  the  contractor  to  determine  corrective 

action  taken. 

9.  Maintain  records  of  all  surveys. 

The  survey  team  will  consist  of  the  INM  or  SupShips  chairman,  a 
representative  of  uhe  cognizant  Bureau  Engineering  group,  a  re¬ 
presentative  of  code  609,  and  one  or  more  specialists  as  may  be 
required  for  the  primary  nature  of  the  contractors  design  relia¬ 
bility  problems.  It  is  extremely  important  that  contractor 
design  reliability  and  analysis  capability  be  surveyed  by  experi¬ 
enced  engineers.  Industry  experience  is  that  "QC"  surveys  cannot 
evaluate  reliability  and  maintainability.  Person-to-person  eval¬ 
uation  of  technical  competence  is  far  more  important  than  check¬ 
ing  documents,  although  the  latter  is  necessary. 

5.2  „  MONTHLY  REPORTS 

The  regular  monthly  contractor  reports  in  chapter  22  section 
12.  serve  as  a  basis  for  monthly  BuShips  evalr&ion  of  reliability 
and  maintainability  (a)  requirements,  (b)  predictions  vs.  appor¬ 
tionment,  (c)  task  progress  vs.  schedule,  and  (d)  verification 
results.  These  must  be  thoroughly  digested  by  the  cognizant  INM 
or  SupShips  reliability  and  maintainability  specialist,  dis¬ 
cussing  any  questionable  items  with  the  contractors  reliability 
and  maintainability  specialist.  Then  for  RDT&E  proqrams  he  can 
prepare  the  reliability  section  of  the  Monthly  Project  Evalua¬ 
tion  (MPE)  and  OPNAVINST  3910.1b  Quarterly  Project  Reliability 
Summary  Sheet  covered  by  BuShips  Instruction  3900.27,  which  are 
then  evaluated  semi-annually  in  accordance  with  BuShips  R&D  memor¬ 
andum  8-64  and  DOD  Instruction  3200.6. 

5.3  RELIABILITY  GROWTH 


It  would  be  very  unrealistic  to  expect  any  contractor  to  achieve 
the  predicted  values  in  the  initial  design.  Predictions  are 
based  upon  more  or  less  mature  designs.  In  the  iterative  process 
of  locating  potential  problem  areas  and  minimizing  their  effects 
or  eliminating  them,  inherent  reliability  improves.  Historically, 
predicted  reliability  increases  during  the  design  phase  and  can 
be  projected  forward  to  operational  use.  As  test  data  becomes 
available,  predictions  have  more  validity.  Problems  invariably 
show  up  in  the  hardware  that  could  not  be  forseen  in  the  design, 
and  achieved  reliability  is  significantly  lower  than  that  inherent. 
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Aggressive  corrective  action  eliminates  these  problems  as  the 
program  progresses,  and  achieved  reliability  increases. 

Analysis  of  reliability  growth  in  the  aerospace  industry  shows 
reasonable  consistency  between  programs,  depending  on  the  degree 
of  urgency  and  its  effect  on  ability  to  implement  corrective 
action  when  problems  are  uncovered  in  tine  hardware  phase.  These 
rates  of  growth  are  roughly  equivalent  to  doubling  the  MTBF  in 
4  years,  and  should  be  applicable  as  approximations  to  ships  GFE 
programs.  Cognizant  Navy  managers  should  be  alert  to  any  signif¬ 
icant  failure  of  such  growth  to  materialize  in  their  programs. 

5.4  POST-AWARD  SURVEYS 

In  order  to  detect  trends  and  assure  compliance  with  reliability 
and  maintainability  requirements,  INM  or  SupShips  should  conduct 
regular  suiveys  of  eacn  contractor  every  o  to  12  months,  covering 
all  BuShips  contracts  he  has  at  each  survey.  Again  the  chapter 
23  Proposal  Evaluation  can  be  used,  chough  it  is  possible  that 
the  excellent  but  very  detailed  NASA  Reliability  Program  Evalua¬ 
tion  Procedures  (2)  will  be  adopted  by  DOD  as  the  government 
standard.  However  the  Procedures  (a)  do  not  cover  several  of  the 
very  important  chapter  23  areas,  (b)  assume  a  particular  ass ign- 
ment  of  organization!  responsibilities  that  many  contractors  may 
find  incompatible  end  uneconomic,  (c)  contain  25  to  50%  redundancy 
for  cross-checking,  and  (d)  seem  very  expensive  to  conduct ,  and 
keep  reasonably  current,  for  the  utility  obtained.  Undoubtedly 
further  refinement  of  it  can  be  expected. 

If  conducted  every  o  months  on  each  contractor,  using  the  same 
chapter  2<  weights,  trends  will  be  evident.  If  the  same  rating 
system  is  used  across  all  contractors,  the  carefully-used  results 
can  highly  motivate  a  contractor  to  correct  deficiencies. 

6.  CONTROL 

Every  effort  should  be  made  to  encourage  the  contractor  to  con¬ 
trol  the  program  himself,  basing  his  decisions  or  recommendat ions 
on  analyses  whicn  are  in  turn  based  upon  the  BuShips  cost.-ef  tec- 
tiveness  criteria.  Then  BuShips  need  only  watch  and  regularly 
evaluate  the  program. 

Often  there  will  be  changes  in  BuShips  planning,  such  as  an  added 
mission  for  a  higher- level  system  that  this  program  supports.  * 

Such  changes  can  easily  change  the  program  cost-effectiveness  * 

criteria,  and  the  contractor  should  he  immediately  so  informed. 
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Then  the  contractor  can  analyse  the  new  situation  and  make  re¬ 
commendations  fcr  contract  change,  if  any. 

But  thee,  there  are  inevitably  contractor  deficiencies  in  achieve¬ 
ment  of  requirements,  achievement  of  tasks,  schedule  adherence, 
or  excessive  expenditure.  It  is  then  up  to  the  SupShips  and/or 
the  Technical  Code  to  (a)  get  ,.he  tacts,  and  thor  ughly  under¬ 
stand  the  problem,  (b)  require  an  official  commit  ent  from  the 
contractor  or  corrective  action  by  a  specified  date,  or  (c)  re¬ 
commend  BuShips  adjustment  of  the  Program  Plan  or  contract  to 
fit  the  unforeseen  contingency. 

Both  technical  and  management  capability  are  vital  to  a  contrac¬ 
tors  ability  to  satisfy  reliability  requirements  within  budget 
and  schedule  constraints.  The  better  the  technical  capability 
the  contractor  has,  the  less  his  attention  is  absorbed  by  the 
question,  "How  can  I  make  this  system  do  the  things  it  should?" 

And  the  more  it  can  be  devoted  to  answering  the  other  question, 
"how  can  I  best  assure  that  th;s  system  will  not  fail?"  Evalu¬ 
ation  of  his  ability  to  meet  reliability  requirements,  therefore, 
involves  not  only  his  understanding  of  organization  for  relia¬ 
bility  and  maintainability  control,  but  his  overall  technical 
background  and  management  history. 

Two  great  obstacles  to  reliability  achievement  are:  (a)  inability 
to  stay  un  schedule,  and  (b)  poor  internal  communication.  As  any 
part  of  the  schedule  begins  to  slip,  pressure  mounts  rapidly  to 
make  do  with  whatever  shortcuts  are  available.  The  first  work 
to  suffer  is  the  review  and  double-checking  type,  with  the  result 
that  errors  are  not  detected.  At  the  same  time,  the  work  is 
done  more  hastily,  so  that  decisions  are  not  weighed  as  carefully 
and  errors  become  more  likely.  By  the  same  token  poor  internal 
communication  increases  the  probability  of  hardware  interface 
problems,  poorly- coordinated  problem  solutions,  and  misunderstood 
requirements . 


7  *  SURVEILLANCE 

InsMat  and  SupShips  are  responsible  for  direct  surveillance  of 
contractor  performance,  and  therefore  of  implementation  of  con¬ 
tractor  reliability  and  maintainability  programs.  Their  resident 
people  will  make  many  tradeoff  decisions  on  problems,  within  their 
authorized  scope,  that  never  need  to  get  to  Washington.  Decision 
by  those  closest  to  the  problem  is  a  tenet  of  good  management. 
There  is  no  substitute  for  close  contact  with  the  contractors. 


•j 
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There  should  be  a  local  InsMat  or  SupShips  reli  abi  lity/ ''maintain¬ 
ability  specialist  who  receives  all  such  contractor  reports,  and 
should  get  Lhem  for  prior  informal  review  to  give  the  con*  r actor 
an  opportunity  to  correct  deficiencies.  He  should  require  monthly 
contractor  meetings  at  which  unresolved  reliability  and  maintain¬ 
ability  problems  are  discussed  and  action  ass ignm.ee cs  made. 

But  beyond  formal  meetings,  he  should  be  continuously  visiting 
appropriate  groups  to  learn  how  things  are  being  done  and  to 
understand  the  problems.  He  should  be  constantly  alert  to 
opportunities  where  he  can  help  the  contractor,  perhaps  by 
getting  information  from  BuShips  or  other  sources. 

7.1  MATERIAL  INSPECTION  SERVICE 

Monitoring  of  progress,  inspection  and  many  of  the  casks  of  con¬ 
tract  administration  are  pe. formed  by  the  Material  Inspection 
Service  (MIS),  including  but  not  limited  to  Inspectors  of  Naval 
Material  (INSMAT)  and  Supervisors  of  Shipbuilding. 

7.2  INSPECTORS  OF  NAVAL  MATERIAL 

INSMAT  offices  are  located  in  large  cit-cs  and  near  major  indus¬ 
tries.  They  work  directly  for  the  Office  of  Naval  Material  and 
have  prime  cognizance  over  source  inspection  of  material ,  pre¬ 
award  and  other  surveys  of  contractors  plants,  and  proper  use  cf 
government  furnished  material  and  facilities.  A  high  level  of 
competence  is  to  be  expected  on  inspection  and  acceptance  tanks 
and  administrat i ve  duties.  Transmittal  of  the  contract  to  the 
local  office  cognizant  over  the  plant  is  all  that  is  required 
to  obtain  their  services  to  the  extent  defined  in  the  contract 
and  speci  f  icat  ions  .  This  means  a  M  unusual  inspect  ion, ••'accept  ance 
functions  must  be  spelled  oat  to  insure  their  aceomp 1 i shment . 

A  high  level  of  engineering  and  inalyticai  competence  is  not  t 
be  expected,  although  it  may  ocas  ion-ally  be  available.  For 
unusual  or  analytical  tasks  that  might  be  considered  within  the 
capabilities  of  a  top  grade  inspector ,  a  letter  should  be  written 
to  the  cognizant  office  outlining  in  conprehens i ve  detail  the 
duties  desired. 

Whore  no  cognizant  INSMAT  u  assigned  over  the  plant,  cross 
servicing  by  Air  Fore?  or  Army  inspectors  may  be  arranged. 

7.3  SUPERVISORS  OF  SHIPBUILDING 


Supervisors  of  Shipbuilding  (SupShips)  arc  BuShips  Inspection 
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Of  lice'?  assigned  cognizance  over  Shipbuilding  and  boat  building 
contracts.  They  are  physically  located  near  the  shipyard  and 
have  the  duties  of  contract  administration  ever  the  details  of 
Hie  contract.  They  have  delegated  contracting  authority  for  the 
negotiation  of  costs  incident  to  changes  authorized  by  the  Bureau, 
they  progreuj  the  work,  enforce  contract  requirements,  and  review 
contractors  drawings  and  procurement.  They  are  staffed  w . th 
Inspection,  Engineering  and  Financial  personnel  with  a  high  level 
of  technical  competence.  At  the  present  time  there  is  no  parti¬ 
cular  capability  in  analytical  reliability. 


Where  special  reliability  analytical  ir  monitoring  tasks  are 
involved  at  the  contractors  plant,  or  in  pro  urement  of  contrac¬ 
tor  furnished  material,  derailed  instructions  should  be  provided 
by  letter . 
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MATERIAL  INSPECTION  SERVICE  UTILIZATION 


Where  reliability  requirements  are  included  in  the  contracts,  the 
cognizant  office  of  the  MIS  should  be  briefed,  preferably  hv 
it  -Lr. ,  of  the  limitations  of  their  authority.  Such  actions  ao 
approval  of  contract  changes,  approval  of  use  of  non-standard 
parts,  and  qualification  by  similarity  should  be  specifically 
assigned  or  specifically  reserved  to  the  Bureau. 
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SUMMARY 


In  this  chapter  we  have  outlined  the  broad  management  of  relia¬ 
bility  and  maintainability  programs,  with  emphasis  on  the  con¬ 
duct,  evaluation,  and  control  of  such  programs.  Control  decisions 
are  made  by  the  Program  Manager  of  RLF&E  programs,  and  by  the 
Ships  Design  Division  or  the  Type  Desk  for  tie  Shipbuilding  and 
Conversion  program. 

Detailed  programs  are  developed  by  the  Technical  Codes  for  tue 
above  approval  and  decisions,  and  likewise  Technical  Code  con¬ 
tract  evaluation  and  control  is  subject  to  the  above  approval 
and  decisions.  Thus  this  chapter  attempts  t)  describe  the  inter¬ 
face  flow  of  work  for  reliability  and  maintainability  programs. 


We  have  tried  to  emphasize  tnat  not  much  will  happen  until  some 
specific  personnel  are  fold  that,  they  "are  henceforth  the  relia¬ 
bility  ard  maintainability  experts,"  that  they  should  ’get  edu¬ 
cated"  on  tne  subject,  and  then  that  they  are  responsible  for 
specific  contract  reliaoility  and  maintainability  achievement. 
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We  have  tried  to  emphasize  that  contract  negotiation  involving 
reliability  and  maintainability  incentives  cannot  be  done  apart 
from  thorough  knowledge  of  reliability  and  maintainability  feasi¬ 
bility,  tasks  and  consequences . 

We  have  tried  tc  emphasize  that  contractor  evaluation  requires 
vachnical  person-to-person  understanding  of  the  problems,  so  that 
the  monthly  reports  have  significance. 

But  abave  all,  if  the  program  is  thoroughly  conceived,  with 
insistence  upon  mutually  understood  Program  Plan  language,  then 
it  really  takes  a  minimum  of  control  to  get  good  results. 
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Chapter  25 
SYSTEM  EFFECTIVES!  .>S 

Admiral  Schoech  recently  stated  (l)i 

System  effectiveness ,  and  its  fiscal  corrollary  cost 
effectiveness  constitute  the  most  important  concern  of 
military  R&D  management." 

System  effectiveness  concerns  the  capability  of  a  system  to  per¬ 
form  its  intended  function.  DOD  Instruction  3200.9  states 

"Trade-offs  shall  be  used  to  obtain,  within  the  mission 
and  performance  envelopes,  an  optimum  balance  between 
total  cost,  schedule  and  operational  effectiveness  for 
the  system." 

The  reasoning  behind  these  statements  is  based  on  high  cost  pen¬ 
alties,  both  direct  and  indirect  which  are  associated  with  low 
reliability.  Some  of  the  direct  costs  are 

-  additional  systems  that  are  required  to  carry  out  a  given 
mission 

-  additional  spares  used  in  support  of  the  systems 

-  added  operating  bases,  supply  and  maintenance  points, 
tenders,  and  test  equipment 

-  additional  maintenance  workload  caused  by  frequent  failures 

-  additional  technical  training  for  maintenance  personnel. 

Some  indirect  causes  related  to  unreliability  which  are  difficult 
to  assess  in  terms  of  cost  are 

-  loss  of  prestige  due  to  failure  of  system 

-  loss  of  ship's  effectiveness 

-  raise  security  which  jeopardizes  America's  defense  posture. 

One  of  the  mc^t  costly  items  attributable  to  unreliability  is  the 
maintenance  and  support  costs  of  systems.  .  .airit enance  expendi¬ 
tures  in  the  DOD  account  for  more  than  25  percent  of  the  defense 
budget.  In  FY61,  960,000  people  (approximately)  were  directly 
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concerned  with  maintenance.  The  figures  quoted  are  believed  to 
have  grown  correspondingly  with  the  defense  budgets.  More  in¬ 
formation  on  the  cost  of  unreliability  is  contained  in  chapter 
26  • 


1 .  THE  BASIS  OF  SYSTEM  EFFECTIVENESS 

The  System  Effectiveness  conce;  *\  is  derived  from  the  system  en¬ 
gineering  process.  It  recognizes  the  interaction  and  interde¬ 
pendence  of  the  many  system  parameters  and  seeks  to  optimize 
them  rationally  in  the  interest  of  overall  system  accomplishment. 
The  concept  commences  with  identification  of  an  operational  re¬ 
quirement.  Ensuing  efforts  are  directed  toward  satisfying  that 
requirement . 

The  selection  of  systems  to  develop  and  the  selections  of  con¬ 
tractors  to  develop  them  must,  be  in  conformance  with  National 
Strategic  Objectives  and  must  include  die  consideration  of  econ¬ 
omic  factors  in  their  acquisition  and  ownership.  That  is  they 
must  be  effective  in  performing  their  function  and  must  make 
most  efficient  use  of  the  budget. 

Reliability  and  Maintainability  must  be  designed  into  equipment 
at  the  system  level,  and  subjected  to  design  and  operational 
trade-off  analyses  with  weight,  size,  cost,  delivery,  etc.  Opti¬ 
mum  system  effectiveness  can  only  result  through  this  judicious 
weighing  of  each  of  the  system  characteristics. 

1.1  APPLICATION 


While  it  is  apparent  that  this  concept  is  clearly  applicable  to 
new  development  programs,  its  use  is  not  limited  thereto.  It 
may  be  applied  equally  well  to  the  Bureau's  problem  of  improving 
existing  systems.  As  with  R&D  programs,  the  concept  is  introduced 
when  operational  factors  are  identified.  The  essential  difference 
is  that  the  concept  can  be  tailored  to  specific  conditions  of 
change  when: 

-  a  previous  operational  requirement  has  been  modified,  or 

-  available  equipment  fails  to  meet  an  existing  operational 
requirement  or 

-  system  improvement  is  desired  to  improve  its  cost-effective¬ 
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The  System  Effectiveness  concept  is  not  restricted  by  the  develop¬ 
ment  nature  of  a  system.  Its  use,  however,  should  be  justified  on 
the  basis  of  its  economy  m  achieving  the  desired  improvement. 
Determinants  may  well  be  the  degree  of  improvement  sought  versus 
constraints  of  time  and/or  cost. 

1.2  CRITERIA  FOR  SYSTEM  EFFECTIVENESS 


The  ultimate  output  of  any  system  is  the  performance  of  a  set  of 
intended  functions.  They  may  be  described  by  some  system  output 
characteristic  such  as  satisfactory  message  transmission  in  a 
communication  system,  or  positive  identification  in  a  shipboard 
radar  system. 

The  system  engineering  approach  views  the  ship  itself  as  a  plat¬ 
form.  Although  it  has  specific  functions,  it  is  dependent  upon 
the  individual  functions  of  its  primary  systems  to  propel,  navi¬ 
gate,  steer  (submerge  for  submarines).  The  function  of  the  plat¬ 
form  (ship)  may  be  to  carry  weapons  or  detection  systems  with 
their  necessary  controls  and  support,  or  to  provide  logistic 
support.  These  systems  require  secondary  or  support  systems  such 
as  personnel,  communications  (internal  and  external),  power,  and 
casualty  control. 

Each  system  has  a  mission  (or  missions)  related  to  either  the 
mobility  and  positioning  of  the  ship,  tne  weapons  or  detection 
equipment,  or  support  of  the  ship's  personnel. 

Mission  Requirements;  Through  proper  evaluation  of  the  system's 
purpose,  it  is  possible  to  establish  mission  duration  require¬ 
ments  for  each  type  of  ship  mission.  Three  standard  mission 
cycles  in  use  are: 

1.  Time  between  shipyard  overhauls  -  4  years. 

2.  Overseas  tour  of  duty,  including  transit  time  -  3  months. 

For  submarines,  submerged  period  may  be  three  months. 

3.  General  Quarters  (battle  stations)-  4  hours. 

Levels  of  Importance:  Systems  may  be  classified  as  to  their 
relative  importance  for  each  of  the  three  cycles.  Fire  Control, 
weapons,  power,  casualty  control,  propulsion,  steering,  etc., 
must  have  a  very  high  Reliability  for  General  Quarters.  However, 
for  a  three-months  tour,  their  essentiality  will  vary.  Propul-  * 

sion  must  have  a  high  Availability,  steering  a  high  Reliability, 
while  fire  control  may  have  no  other  requirement  than  stand-by 
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readiness.  Search  equipment  may  requ  re  a  high.  Availability. 

Each  system  must  1 '  evaluated  against  its  requirements,  including 
requirements  to  support  other  systems  in  order  to  establish  mean¬ 
ingful  essentiality  levels. 

Capability  of  Restoration:  The  capability  of  self  maintenance  of 
snips  does  in  fact  have  limits-.  The  limitations  are: 

1.  skills  of  personnel 

2.  availability  of  personnel 

3.  spare  parts  and  materials 

4.  operational  constraints 

5.  equipment  and  facilities 

6.  access. 

Summary :  The  term  System  Ef fectiveness  is  used  to  describe  the 
overall  accomplishment  or  worth  of  a  system.  It  relates  to  that 
property  of  system  output  which  is  the  reason  for  its  existence  - 
namely,  the  carrying  out  of  some  intended  function.  The  System 
Effectiveness  concept  realistically  considers 

1.  system  function 

2.  system  mission  duration  requirements  based  on  stated 

cycles  and  logistics  capability 

3.  levels  of  system  importance 

4.  capability  of  restoration. 

If  the  system  is  effective,  it  will  carry  out  its  function  for 
the  duration  under  actual  conditions.  If  it  is  not  effective, 
attention  must  be  given  to  the  system  parameters  which  are 
deficient . 


2  *  SYSTEM  EFFECT IV ENESS  FACTORS 

System  Effectiveness  is  the  combination  of  many  faclois,  each  in 
some  way  contributing  to  the  capability  of  the  system  to  perform 
its  intended  function.  Some  of  these  are: 

A.  Performance  Capability 

1 .  Technical  Capability 

a.  Capacity  (load,  range,  etc.) 

b.  Speed  (knots,  microseconds,  etc.) 

c.  Accuracy  (bearing,  resolution,  etc.) 

d.  Invulnerability  to  countermeasures 
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2 .  Possible  Limi'ations  Upon  Performance 

a.  Space  and  weight  requirements 

b.  Input  power  requirements 

c.  Input  information  requiremer.  s 

d.  Requirements  for  special  protection  against  shock, 

radiation,  vibration,  high  pressure,  and  other 
environmental  influences 

B.  Dependability 

1 .  Reliability 

a.  failure- free  operation 

b.  Redundancy  or  provisions  for  alternate  modes  of 

operation 

2 .  Maintainability 

a.  Time  to  restore  failed  systems  to  satisfactory 
operating  status 

b  Technical  skills  required  for  maintenance 

c.  Effects  of  use  upon  maintenance 

3 .  Logistic  Supportability 

a.  Spares  availability 

b.  Test  equipment  and  facilities 

C .  Procurability 

1.  Acquisition  Cost 

2.  Development  Time 
2.1  PERFORMANCE  CAPABILITY 


The  reason  for  the  existence  of  the  ship  is  the  performance  of 
an  intended  function.  With  the  large  number  of  possible  require¬ 
ments  that  may  be  operationally  imposed  on  a  ship,  the  equipment 
installed  can  not  always  be  matched  to  the  requirements.  A  radar 
operating  within  specifications  may  i  ot  be  able  to  detect  a 
surface  target  at  the  extreme  of  its  range  (due  to  heavy  weather), 
or  it  may  not  be  able  to  detect  a  target  beyond  the  limit  of  its 
range  (we  might  call  this  mis-application) .  Or  the  requirement 
w  iy  not  utilize  the  full  capability  of  the  equipment,  as  a  twenty- 
f.'.ve  megaton  bomb  greatly  overkilling  a  target. 


To  start  to  evaluate  the  effectiveness  of  an  equipment  for  a 
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particular  function  or  "mission",  the  first  need  is  a  figure  of 
merit  for  the  proposed  (or  each  proposed)  system  that  describes 
the  limit  of  its  capability.  Many  such  figures  of  merit  can  be 
envisioned.  For  a  supply  ship,  tc  take  a  simple  case,  such  a 
figure  of  merit  could  be  the  ton-miles  of  stores  it  could  haul, 
the  ton  being  based  on  full  capacity  loading  and  the  miles  on 
maximum  cruising  radius  without  refueling.  This  figure  of  merit 
we  call  Performance  Capability. 

For  a  specific  mission  this  capability  may  or  may  not  be  adequate. 
Where  it  is  not,  the  system  cannot  be  considered  for  that  parti¬ 
cular  mission.  Comparisons  of  excess  capability,  where  the  pro¬ 
posed  system  is  more  than  adequate,  become  an  area  of  trade-off 
with  cost,  delivery  schedule  and  other  factors. 

2.2  DELIVERY  EFFECTIVENESS 

As  stated  earlier  in  chapter  2,  the  development  of  new  systems 
is  always  based  on  a  General  Operational  Requirement.  The  mid¬ 
range  strategic  objectives  normally  provide  a  time  that  the 
equipment  is  needed  in  the  '"leet.  The  length  of  development 
time  can  and  does  control  the  amount  of  Performance  Capability, 
as  previously  used,  that  can  be  developed  into  a  proposed  system. 

For  any  particular  delivery  schedule  there  is  some  limit  to 
achievable  capability.  If  more  time  can  be  allowed  a  higher  level 
of  capability  might  be  developed.  Again,  if  a  capability  adequate 
to  perform  the  intended  function  on  a  particular  mission  cannot 
be  provided  within  time  constraints,  the  proposed  system  cannot 
be  considered  as  supporting  that  operational  requirement. 

Where  excess  capability  can  be  provided  then  Performance  Capabil¬ 
ity  and  schedule  can  be  traded-off  in  selecting  the  capability 
to  develop  into  the  system. 

There  is,  of  course,  always  some  risk  involved  that  the  equipment 
will  not  be  delivered  on  schedule.  This  is  especially  true  when 
major  technological  breakthroughs  are  required,  but  can  also 
occur  in  standard  production  contracts  such  as  when  major  strikes 
occur.  The  penalty  for  failure  to  have  the  equipment  operable 
must  be  assessed  (in  terms  possibly  of  importance  of  meeting  the 
particular  objective  within  the  time  frame,  or  adequacy  of  pre¬ 
sent  interim  systems  to  achieve  the  military  purpose).  The  cap¬ 
ability  to  be  developed  into  the  system  must  be  selected  such 
that  the  risk  of  late  delivery  is  reduced  to  an  acceptable  value. 

Within  the  constraints  provided  (mission  and  performance  envelopes) 
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a  relationship  between  the  effectiveness  (usefulness)  and  probable 
delivery  schedules  may  sometimes  be  found.  In  some  cases  this 
will  be  bounded,  that  is,  limited  on  one  or  both  ends  of  a  del¬ 
ivery  period.  If  for  example  the  system  were  a  fire  control  for 
a  new  weapon,  delivery  before  the  weapon  become  available  would 
serve  little,  if  any,  purpose. 

Or  if  the  equipment  were  for  interim  use  until  a  new  system  with 
far  superior  capabilities  was  developed,  delivery  too  close  to 
the  phase-out  date  is  obviously  uneconomical.  In  most  cases  the 
relationship  will  be  highly  subjective.  The  "decision  maker" 
will  have  to  assign  values  to  the  various  possible  combinations 
of  capability  and  delivery  to  use  in  determining  the  desirability 
of  the  various  possible  courses  of  action  (various  competing 
systems ) . 

2.3  UTILIZATION 


As  earlier  mentioned,  the  capability  of  the  equipment  may  be  in 
excess  of  requirements.  The  excess  capability  is,  in  effect 
wasted  --  that  is  serves  no  useful  purpose.  To  describe  the 
useful  capability  we  can  employ  a  Utilization  factor,  a  fraction 
describing  what  portion  of  the  available  capability  is  used  in 
the  particular  mission  being  considered. 

Due  to  environmental  effects  the  capability  may  at  times  be  less 
than  the  requirements.  In  this  mse,  we  can  consider  the  Utili¬ 
zation  factor  as  the  fraction  of  time  that  the  equipment  is  capable 
of  achieving  the  requirement  within  the  limitations  imposed  by 
effects  external  to  the  system,  even  when  the  system  is  perform¬ 
ing  within  specifications.  The  Utilization  factor  reflects  re¬ 
duction  in  capability,  as  well  as  use,  due  to  influences  external 
to  the  system.  The  product  of  the  Performance  Capability  index 
and  the  Utilization  factor  provides  a  measure  of  usable  capabil¬ 
ity  within  the  mission  requ i rement . 


Many  systems  have  a  spectrum  of  possible  mission  requirements. 
The  most  economical  set  of  systems  is  probably  (but  not  always) 
the  minimum  number  of  systems  that  meet  all  the  mission  require¬ 
ments.  So  each  proposed  system  must  be  tested  against  all  per¬ 
tinent  requirements  and  a  system  selected  that  optimizes  the  use 
of  funds  (within  per formance  and  delivery  schedule  constraints) 
with  due  consideration  of  phase-out  of  other  systems  and  intro¬ 
duction  of  other  systems  under  development. 
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2.4  DEPENDABILITY 


The  Performance  Capability  of  a  system  (when  working  right)  is 
useless  if  the  equipment  is  not  oper  ole  when  it  is  required. 
Failures  of  the  equipment  during  the  attempt  to  perform  the  re¬ 
quired  function  may  prevent  the  accomplishment  of  the  mission. 

The  dependability  of  the  equipment  —  its  operability  when  re¬ 
quired  and  its  reliability  when  operating  —  can  influence  the 
effectiveness  of  the  equipment.  Two  factors  are  involved. 

These  are  normally  termed  Reliability  and  Availability. 

Availability,  with  certain  exceptions,  represents  the  readiness 
of  the  system  to  respond  on  demand.  Reliability  is  the  ability 
of  the  system  to  operate  for  the  required  period,  provided  it 
was  capable  of  responding  on  demand.  The  dependability  computa¬ 
tion  depends  on  the  nature  of  the  requirements. 

2.4.1  Simple  Reliability:  Let's  consider  a  case  such  as  the 
steering  engines  of  a  ship.  The  requirement  for  the  steering 
engines  is  "no  failure"  during  a  particular  period.  Every 
command  for  rudder  angle  must  be  obeyed  --  no  down  time  is 
tolerable.  The  use  is,  in  effect,  continuous  while  underway. 

The  consequences  of  failure  to  respond  with  a  change  of  rudder 
angle  are  considered  unacceptable.  For  this  example  the  Effec¬ 
tiveness  of  the  equipment  is  less  than  the  (used)  Performance 
Capability  by  a  factor  based  on  the  probability  that  it  will  con¬ 
tinue  operating  daring  the  required  operating  time.  This  factor 
is  the  Reliability,  R.  Where  the  Performance  Capability  is  P,  and 
the  Utiliz  ation  factor  is  U,  the  Effectiveness  E  is  the  product: 

E  =  PRU 

In  this  case.  Availability  is  not  meaningful  since  the  consequences 
of  failure  are  considered  identical  regardless  of  downtime. 

2.4.2  Availability:  Consider  again  a  radar,  this  time  a  search 
radar  whose  sole  purpose  is  to  detect  (but  not  track)  approaching 
"bogies" .  The  detection  and  reporting  are  essentially  instan¬ 
taneous  .  Once  detected  and  reported,  the  approaching  aircraft 

is  assigned  to  a  second  radar  set  to  track.  As  soon  as  the  "bogie" 
is  reported  there  is  no  further  requirement  on  this  radar  in 
connection  with  this  target.  The  operating  time  for  this  radar 
is  continuous  for  a  three  month  cruise.  In  this  case,  the  Effec¬ 
tiveness  of  the  radar  is  the  Performance  Capability  multiplied 
by  its  Utilization  and  by  its  Continuous  Availability  Ac,  which 
in  turn  is  the  probability  that  it  is  operable  at  any  time  during 
the  three  months  cruise.  That  is: 
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E  -  PUAC 

The  Reliability  is  considered  not  pertinent  to  the  requirement, 
except  as  reliability  parameters  influence  the  /..'ailability . 

2.4.1  Combined  Reliability  and  Availability;  Assume  a  system 
such  ->s  a  fire  control  radar.  The  requirement  for  the  system  is 
to  lock  'n  to  a  target  --  if  and  when  a  target  appears  --  and 
direct  the  weapon  in  hitting  the  target.  The  demand  can  come 
once  a  day  or  once  a  month.  Two  things  are  important,  --  that 
(a)  the  equipment  be  ready  to  respond  to  a  demand  if  it  comes  and 
that  (b)  once  the  demand  is  made  and  the  equipment  starts  to 
operate,  that  no  failure  prevents  the  fulfillment  of  the  require¬ 
ment  . 


Trie  Effectiveness  of  this  equipment  is  its  Performance  Capability 
multiplied  by  the  probability  that  it  is  "ready"  when  required 
(the  Availability,  Ac),  and  then  multiplied  by  the  probability 
that  it  will  perform  successfully  (the  Reliability,  R) .  In  this 
case,  the  Effectiveness  is  the  prod  ct 


E  -  PACRU 

It  is  important  to  note  here  that  where  the  Reliability  of  the 
equipment  is  different  in  the  standby  mode  than  in  the  operating 
mode  --  that  is,  where  more  failures  or  more  frequent  failures 
are  to  be  expected  when  the  equip-,.  at  is  operating  than  when  it 
is  standing  by,  the  value  •  -  f  Availability  should  be  computed 
based  on  the  equipment  MTBE  in  trie  standby  mode. 

Another  illustration  of  this  situation  is  a  radar  set  whose 
function  is  to  detect  and  track  targets.  In  per  forming  detection, 
only  t!ie  r  idar  set  is  used  to  provide  signals  on  a  scope.  When 
a  si.aal  appears  on  the  scope,  a  computer  is  turned  on  and  a  signal 
controlled  by  a  human  operator  manipulating  buttons  and  a  cursor 
on  the  scope  provide  input  to  the  trucking  computer. 


For  the 

complete  equipment  the 

cecer  at 

Mode  A  : 

mi  tracking.  Mode  F. 

Si  nee  t 

p  'r  t  i  on 

of  the  equipment  is  ne 

cess  try 

the  e  ffect  iversess  of  th»  equipment  is 
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for  trio  tracking  operation, 
the  capability  used  (FU) 
ability  in  mode  A  ( A  A •  and 
That  is: 


A+B 


2  5-11 


2.4.4  Summary •,  As  may  be  noted  by  examining  the  different 
situations,  the  dependability  factor  is  determined  by  the  way 
the  requirements  are  stated  Where  the  primary  requirement  is 
"no  failures",  reliability  is  the  primary  consideration.  Where 
the  requirement  is  maximum  "up  time"  Availability  becomes  the 
ci iter  ion.  Where  the  requiied  operation  is  more  complex,  a 
combination  of  Availability  and  Reliaoility  is  needed  to  measure 
the  dependability. 

A  more  general  statement  of  Eltectiv^ness  would  include  factors 
such  as  Delivery  Effectiveness,  D,  in  the  general  relationship 
E  =  f(P,U,  D,R,M)  where  the  function  f(x)  merely  states  that  the 
Effectiveness  is  some  relationship  between  the  factors  to  be 
determined  from  the  stated  i i rements . 


3 .  RELIABILITY  IN  SYSTEM  EFFECTIVENESS 

As  we  have  seen,  one  of  the  primary  parameters  of  system  Effec¬ 
tiveness  is  the  Reliability.  Of  tnese  interrelated  parameters, 
Reliability  is  the  most  susceptible  to  expression  out  of  context. 
However,  even  in  reliability  studies,  an  investigation  cai not  be 
conducted  unless  it  is  extended  to  include  related  characteris¬ 
tics  that  influence  system  Effectiveness. 

It  has  long  been  apparent  that  the  maximum  °  '  ability  that  can 
be  achieved  by  any  system  is  the  amount  designed  into  the  equip¬ 
ment.  Therefore,  the  requirements  for  design  must  be  carefully 
apportioned  among  the  various  pieces  of  equipment  and  controlled 
throughout  the  design  effort.  Any  work  accomplished  to  make  tie 
design  a  hardware  reality  will  tend  to  reduce  the  designed  Relia¬ 
bility  if  these  factors  are  not  carefully  controlled. 

Figure  25-12  depicts  most  of  the  areas  where  in  careful lv  planned 
control  must  be  exercised  in  order  to  prevent  undue  degradation 
of  Reliability.  It  also  describes  the  action  which  must  be  taken 
by  the  Bureau  of  Ships  in  order  for  it  to  adequately  control  those 
areas  for  which  it  is  responsible.  It  immediately  becomes  evident 
that  for  high  operational  reliability  tc  be  obtained,  close  team¬ 
work  is  necessary  between  the  Bureau  of  Ships  and  the  contractor. 
Furthermore,  top  management  must  be  aware  of  and  support  all 
pertinent  elements  necessary  in  a  thoroughly  coordinated  manner 
in  order  to  achieve  high  operational  Reliability. 

Operational  reliability  is  one  measure  of  system  Effectiveness. 

For  equipment  which  could  operate  continuously  or  upon  demand 
without  failing,  this  measure  would  be  100  percent.  This  figure 
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can  at  best  only  be  approached  asymptotically  as  the  art  of 
system  design  advances. 

3.1  ESTABLISHING  REQUIREMENTS 


One  approach  tc  establishing  a  requirement  is  to  determine  the 
unreliability  which  we  are  willing  to  tolerate  in  a  system. 
Obviously  if  its  function  is  critical  to  mission  success,  then 
we  will  tolerate  relatively  few  failures.  From  this  approach, 
we  can  derive  a  principle  which  is  one  basis  for  establishing  a 
reliability  requirement  --  namely,  that  the  relative  importance 
of  the  system  to  the  ship's  mission  is  influential  in  determining 
the  assigned  requirement.  The  requirement  may  be  expressed 
simply  in  terms  of  the  failures  permitted  during  a  given  mission 
typie  and  duration  under  specified  conditicns. 


Following  this  reasoning  a  step  further,  why  not  merely  state 
that  no  failures  will  be  permitted,  so  that  we  have  100  percent 
reliability  of  the  system  during  the  mission?  This  reliability 
is  probably  unattainable  and  impractical  even  as  a  design  goal. 

In  design  of  a  ship,  system  or  part,  reliability  is  in  competition 
with  Performance  Capability,  cost,  time,  logistics,  and  obsoles¬ 
cence.  The  latter  would  probably  be  attained  prior  to  100% 
Reliability  even  if  all  other  parameters  were  most  favorable  to 
reliability . 


This  is  not  to  say  that  extremely  high  reliability  cannot  be 
achieved.  It  can,  but  often  it  cannot  be  realized  economically, 
or  without  undue  penalty  upon  performance  or  schedule.  We  know 
from  experience  that  ieliability  of  a  system  can  often  be  in¬ 
creased  significantly  within  the  state-of-the-art  using  the 
approaches  given  in  chapter  13.  The  improvement  to  reliability 
is  usually  limited  by  cost,  schedule,  etc. 


3.2  METHOD  OF  ANALYSIS 


In  the  application  of  reliability  principles,  to  determine  oper¬ 
ational  reliability  requirements,  it  is  necessary  to  identify 
the  events  which  comprise  a  system  activity  cycle.  For  ships, 
the  activity  cycle  consists  of  various  types  of  operational 
phases.  The  logical  sequence  of  events  considered  for  the  anal¬ 
ysis  of  operational  reliability  is  shown  in  Figure  25-14.  The 
operational  period  for  each  system  is  from  system  turn-on  through¬ 
out  the  operational  period  to  system  turn-off.  For  some  systems 
(propulsion)  this  may  include  the  complete  Underway  period.  For 
other  systems  (communications)  this  includes  both  At  Anchor  and 
Underway.  For  still  others  (weapons)  this  may  include  only 
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Battle  Condition  with  readiness  requirements  for  patrol  or  war 
zone  cruising. 

Reliability  parameters  contained  in  each  operational  phase  are 
mission  required,  system  required,  and  use  factors.  For  a  parti¬ 
cular  type  of  operation,  this  would  include  the  elements  require- 
ing  reliability  measures  in  preparing  for  sea,  transition  to  an 
operating  area,  on  station  or  patrol,  engagement  action,  transi¬ 
tion  to  port.  Startup  and  checkout  of  each  system  are  considered 
part  of  the  operation  and  the  reliability  measure  includes  this 
operation  time. 

The  extent  of  unscheduled  maintenance  performed  both  at  anchor 
and  underway  is  dependent  upon  the  requirements  for  the  present 
and  future  operational  phases.  Scheduled  maintenance,  both  a-L 
anchor  and  underway,  includes  major  maintenance  functions  of 
periodic  inspections,  minor  inspections,  and  modification  of 
equipment. 

3.2.1  Operational  Reliability  Analysis;  In  the  operational 
phase  of  the  reliability  analysis,  all  of  the  events  are  con¬ 
sidered  from  deployment  order  to  systems  shutdown  after  the 
operational  phase  is  over.  Many  variables  are  involved  within 
each  operational  phase,  such  as  different  missions  requi"ed,  var¬ 
iations  of  operational  parameters  within  each  mission  type,  diff¬ 
erent  preparation- for-sea  reliability  values,  and  variability  of 
mission  success  criteria.  However  a  complete  general  analysis  to 
determine  operational  phase  reliability  for  ^ny  combination  of 
the  variables  is  possible.  The  method  of  attack  is  to  select 
the  design  mission  for  an  extensive  analysis  and  then  to  carry 
the  analysis  over  to  other  missions  to  the  extent  possible. 

To  reduce  the  amount  of  computation  necessary,  only  a  finite 
number  of  significant  points  within  the  mission  profile  is  usually 
selected  to  measure  actual  reliability  values.  These  points  may 
be  the  equipment  required  for  in  port  use,  the  equipment  necessary 
for  the  on  station  or  patrol  phase,  equipment  necessary  for  en¬ 
gagement  action,  and  transition  back  to  port.  Due  consideration 
must  be  given  to  on-board  maintenance  capability  and  the  opera¬ 
bility  time  r  all  equipment. 

* 

One  of  the  most  important  uses  of  the  system  efi._  eness  con¬ 
cept  is  to  make  it  possible  for  the  maintenance  support  planner 
to  obtain  a  reliability  measure  for  a  mixed  family  of  missions 
from  inherent  failure-rate  principles.  These  measure  of  relia¬ 
bility  indicate  the  failure-induced  frequency  for  maintenance. 
However,  the  maintenance  support  planner  knows  from  experience 
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that  failures  occur  which  are  not  induced  solely  by  equipment 
design  ch aracteristics .  The  ratio  between  the  total  failures 
requiring  maintenance  acvion  to  failures  induced  by  equipment 
design  only  is  actually  greater  than  2  to  1.  However,  system¬ 
atic  approaches  are  available  which  can  be  applied  to  derive  the 
conversion  ratio  of  MTBF  to  MTBM . 

MTBF/MTBM  Relation;  To  clarify  the  problem  of  converting  MTBF 
to  MTBM,  it  is  necessary  to  consider  the  basic  principles  of 
reliability.  The  characteristic  failure  rate  curve  of  equipment 
is  shown  in  Figure  25-17.  The  random  period  of  equipment  opera¬ 
tion  contains  not  only  the  design  failure  rate  but  others  such 
as  misuse  of  equipment  by  the  operator,  maintenance  induced,  and 
operation  ’ r.  an  environment  outside  the  original  specification.. 
These  misuses  of  equipment  can  also  increase  the  normal  random 
failure  rate. 

The  separation  of  MTBF  and  MTBM  on  the  life  characteristic  c.’rve 
generates  changes  in  both  failure  distributions  and  reliability 
measures  as  shown  in  Figure  25-17.  It  is  readily  seen  that  the 
failure  rate  of  MTBM  equals  K  times  the  MTBF  failure  rate  where 
K  equals  MTBM  divided  by  MTBF. 

In  the  failure  distribution  graph,  MTBM  is  seen  to  have  a  higher 
frequency  of  failures  with  respect  to  time  than  MTBF.  It  is  re¬ 
called  that  frequency  is  equal  to  the  reciprocal  of  MTBF.  In 
like  manner,  the  frequency  of  MTBM  is  a  reciprocal.  Since  MTBF  is 
greater  than  MTBM,  it  follows  that  for  any  one  point  of  operating 
time  the  MTBM  induced  frequency  is  greater  than  the  frequency 
generated  by  MTBF. 

The  MTBF  and  MTBM  are  shown  in  Figure  2  5-17.  As  expected,  the 
probability  of  equipment  being  operable  without  a  maintenance 
action  is  less  than  the  probability  of  operating  without  a 
failure . 

3.2.2  study  Approach;  Reliability  studies  for  systems  use  a 
dynamic  approach  by  expanding  the  reliability  concept  to  determine 
measures  which  can  be  used  to  evaluate  the  effects  of  operational 
activities,  such  as  in-port  use,  operational  readiness,  and  main¬ 
tenance,  or  equipment  operability.  In  Figure  25-18  the  study 
approach  to  system  reliability  is  summarized.  Corresponding 
design  feedback  loops  are  included.  As  noted,  the  study  of  re¬ 
liability  bey  ins  with  the  design  concept.  Throuyh  reliability 
measurement  functions,  an  MTBF  is  obtained  for  each  equipment 
for  the  different  modes  of  operation.  This  MTBF  measure  is  used 
to  develop  a  composite  MTBF  measure  for  operational  reliability. 
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The  .step  between  Mean-T  ime-Between-Ma  i  nt  cnance  action  (MTBM)  is 
the  relationship  between  reliability  and  maintenance.  Reliability 
dictates  the  non-schedu led  maintenance  iob  ar.d  indirectly  the 
scheduled  maintenance  job.  Maintenance,  in  turn,  determines  the 
degree  of  reliability  restoration,  based  upon  the  efficiency 
with  which  the  maintenance  job  is  accomplished. 


4 •  RELIABILITY  AND  MAINTAINABILITY  IN 

SYSTEM  EFFECTIVENESS 


The  achievement  of  the  desired  objective',  capability  of 
successful  operation  of  equipment,  demands  a  dynamic  reliability 
proqram  which  serves  to  control  the  development  and  design  of  the 
system  from  the  conceptional  phase  throughout  its  production  and 
operation.  Both  the  Bureau  of  Ships  and  contractors  are  making 
proqress  in  increasinu  the  validity  and  application  of  reliability 
integration  processes.  The  importance  of  reliability  in  the 
system  program  is  depicted  in  Figure  25-20.  As  shown  the  relia¬ 
bility  measure  of  Mean  Time  Between  Failure  (MTBF)  is  determined 
by  design  computation.  This  measure  denotes  the  operability  and, 
conversely,  the  inoperability  of  equipment.  Inoperability  means 
that  equipment  has  failed  and  the  failure  has  resulted  in  system 
downtime  and  the  need  for  logistic  support. 


Because  of  its  effect  on  equipment  operability,  system  downtime, 
and  logistics,  reliability  is  a  leading  contributor  to  an  effec¬ 
tiveness  measure.  In  operational  terms,  it  can  be  said  that  the 
measure  of  operational  readiness,  redeployment  rates,  success  of 
operator,  maintenance  specialists,  test  equipment,  spares,  etc., 
are  directly  dependent  upon  the  measure  of  equipment  reliability 
and  indirectly  affect  the  scheduled  maintenance. 

We  will  consider  only  briefly  the'  trade-off  relationships  between 
the  Effectiveness  measures,  Reliability  and  Maintainability,  as 
a  function  of  system  Availability.  Availability  is  considered 
to  be  the  operational  time  divided  by  the  total  time,  i.e.. 


Ava i 1 abi 1 i ty 


t ot a  1  oper  at i ona 1  t  ime 
tot  a  1  t ime 


For  example,  i f  we  had  a  system  committed  to  an  operation  for  30 
days,  the  total  time  would  be  30  x  24  or  720  hours.  If  during 
t.nis  periou,  the  system  were  down  for  corrective  maintenance  for 
a  total  of  48  hours,  the  Availability  of  the  system  would  then  be: 


720  -  48 


7  20 


.92  . 
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It  should  be  apparent  that  the  restoration  time  of  48  hours  can 
be  caused  by  many  factors. 

Availability  may  be  portrayed  conceptually  as  follows; 


AVAILABILITY 


1  RELIABILITY  MAINTAINABILITY 

Time  to  failure  Time  to  restore 

Availability  can  be  described  as 

MTBF 

A  ’  MTBF  +  M1TR 

where  MTTR  =  Mean  Time  to  Restore,  and  consequently,  foi  any 
given  Availability,  is  directly  porportional  to  the  Mean  Time 
Between  Failure.  For  a  given  Availability,  this  implies  that  by 
doubling  the  MTBF,  we  can  accept  twice  the  maintenance  time. 

A  specified  or  required  Availability  can  be  obtained  from  many 
different  combinations  of  Reliability  and  Maintainability.  Con¬ 
sequently,  the  allocation  of  fixed  resources  should  dere,.d  upon 

(a)  the  ease  and  relative  cost  of  increasing  these  elements  and 

(b)  the  increase  in  Availability  accompanying  specified  increases 
in  each  element  or  combination  therein. 

Within  the  limitations  previously  discussed,  Reliability  and 
Maintainability  can  be  used  to  substitute  for  each  other  over  a 
wide  range  of  Availability  in  a  complex  system.  Efforts  within 
the  limitations  of  economic  and  technical  constraints  may  be 
allocated  to  develop  more  or  less  Reliability,  and  relatively  more 
or  less  Maintainability  in  attaining  either  increases  in  Avail¬ 
ability  or  minimum  cost  levels  of  Availability.  Therefore,  the 
substitution  of  Reliability  for  Maintainability  becomes  an  appro¬ 
priate  concept  for  decision-making  purposes. 

5.  SYSTEM  EFFECT  IVENESS  MODELS 

Reliability  models  are  used  to  explore  and  validate  alternate 
preliminary  design  approaches.  Models  may  be  used  to  study  com¬ 
plexity,  predict  Reliability  achievable  'n  design,  predict  ulti¬ 
mate  Availability  based  upon  Reliability  and  Maintainability,  and 
to  determine  the  growth  of  Reliability  during  the  design.  In  a 
sense  such  models  are  limited  to  relating  Reliability  and  Main¬ 
tainability  as  system  design  parameters.  In  this  chapter  we  see 
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a  broader  use  of  the  system  Effectiveness  models  to  establish  a 
ship's  mission  Effectiveness  requirement  in  ferms  of  operational 
and  deployment  factors,  such  as  readiness  rates,  mission  mode 
Reliability  and  Maintainability.  The  latter  uses  are  currently 
being  expanded  to  include  all  other  elements  related  to  Effective¬ 
ness,  including  cost  and  schedule. 

It  now  appears  that  such  a  full  effectiveness  concept  is  feasible, 
and  evolutionary.  Techniques  are  being  developed  for  quantifying 
the  remaining  elements  so  that  computer  technology  may  be  employed 
to  develop  a  single  measure  that  i^  based  upon  optimization  of  the 
fundamental  measures  of  total  operational  Effectiveness.  The 
rapid  development  of  this  total  operational  Effectiveness  concept 
is  a  worthy  goal  of  managers  interested  in  naval  efficiency 
{refer  to  Figure  25-23). 

The  procurement  of  costly  complex  systems  is  a  risky  business. 
Despite  all  efforts  to  thoroughly  assess  proposals  prior  to 
award,  the  contractor  selection  process  is  still  far  from  ideal . 
Vagueness,  particularly  with  regard  to  the  ultimate  Effectiveness, 
is  characteristic  of  most.  If  the  award  is  based  upon  lowest 
Acquisition  Cost,  it  could  well  prove  to  be  the  most  costly  system, 
unless  the  parameters  other  than  cost,  schedule,  and  performance 
are  clearly  treated  with  the  importance  they  deserve.  See  chapter 
23,  section  5.1.2,  and  chapter  26. 

The  use  of  models  as  a  means  of  validating  "proposal  promises" 
could  improve  the  contractor  selection  process.  This  method 
could  be  used  by  requiring  the  submittal  of  a  model  as  part  of 
the  proposal.  If  such  a  model  clearly  substantiates  a  proposed 
approach  as  a  valid  one,  the  accepted  model  values  can  be  invoked 
in  specifications.  The  model  can  be  used  to  determine  the  buyer 
and  seller  risks  associated  with  demonstrating  Reliability  and 
Maintainability  values  when  demonstration  and  correction  of  de¬ 
fects  are  requireu  under  Fixed  Price  contracts.  The  model  can  be 
an  invaluable  tool  in  controlling  the  achievement  of  reliability 
during  a  long  design  and  development  program  where  numerous  changes 
are  introduced.  The  effect  of  such  changes  on  Reliability  can  be 
determined  at  any  "break- in"  point.  It  serves  ilso  as  a  built-in 
Reliability  and  Maintainability  audit  apparatus  that  the  Contrac¬ 
ting  Officer  can  examine  at  any  time. 

Through  comparison  of  input  allowances  for  degradation  nth  the 
actual  degradation,  the  mode’  can  also  be  used  to  determine  the 
control  of  reliability  subsequent  to  design.  As  a  fina’  step, 
it  obviously  can  assess  the  achievement  of  reliability  and  main¬ 
tainability  by  comparing  the  require  nents  vi i n  rhe  operational 
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GENERATION  OF  SYSTEM  REQUIREMENTS 


EFFECTIVENESS  PERFORMANCE 
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experience . 

While  the  models  and  computers  are  extremely  useful  tools,  they 
do  not  replace  Navy  management.  Neither  do  they  substitute  for 
good  reliability  design  practices,  or  quality  control.  Despite 
the  wonders  of  the  computer,  and  of  Reliability  Engineering  as 
the  science  of  excellence,  we  are  still  dependent  upon  Admirals, 
civil  servants,  seamen,  and  contractors  to  provide  a  ships  worth 
of  confidence  in  every  ship. 
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Chapter  26 
COST-EFFECTIVENESS 


This  chapter  is  concerned  with  a  very  old  problem .  Just  how  old, 
and  how  universal,  is  apparent  in  this  quotation  from  Shakespeare 
in  King  Henry  IV,  part  2.  Act  I,  scene  3: 

. When  we  mean  to  build, 

We  first  survey  the  plot,  then  draw  the  model; 
and  when  we  see  the  figure  of  the  house, 
then  must  we  rate  the  cost  of  the  erection; 
which  if  we  find  outweighs  ability, 
what  do  we  then  but  draw  anew  the  model 
in  fewer  offices,  or  ax,  last  desist 
to  build  at  all? 

For  the  last  several  years  we've  had  a  shiny  new  name  for  the 
same  old  solution  to  the  same  old  problem.  We've  called  it  'cost- 
effectiveness."  But  let's  have  a  look  at  the  modern  problem. 

1.  OPERATIONAL  COST  EXPERIENCE 

During  World  War  II,  with  its  unprecedented  dependence  upon  elec¬ 
tronic  gear,  the  consequences  of  unreliability  became  painfully 
apparent.  After  the  war  many  military  surveys  were  made  to  eval¬ 
uate  the  problem,  and  the  new  reliability  technology  was  developed 
under  forced  draft. 

But  the  problem  did  not  go  away.  It  became  obvious  to  our  military 
leaders  that,  the  great  cost  of  unreliability  was,  and  still  is, 
using  a  very  large  share  of  our  defense  dollar  resources.  Funds 
needed  for  more  or  better  weapon  systems  are  being  pre-empted  by 
high  maintenance  cost,  and  sometimes  by  ill-considered  develop¬ 
ment  and  premature  production.  Here  are  some  quotations  of  our 
military  leadership: 

1.1  DOD  EXPERIENCE 


Rober t  S.  McNamara,  Secretary  of  Defense,  28  March  1963  speech 
to  the  Senate  Joint  Economic  Council 

"All.  too  often  large-scale. .  .developments ,  and  even  production 
programs,  have  been  undertaken. . .before  we  had  clearly  deter¬ 
mined  that  there  existed  a  suitable  technological  base,.... 
what  it  would  cost, . . . .and  wherher  the  capability  would  be 
worth  the  cost.  As  a  result ,.. .changes  are  being  made."  One 
change  is  DOD  directive  3200.9,  excerpted  in  Figure  26-3. 
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Charles  J.  Hitch,  Comptroller ,  Dept,  of  Defense  ( 2 ,  p .  6  3 ) 

"Military  requirements  are  meaningful  only  in  terms  of  bene¬ 
fits  to  be  gained  in  relation  to  their  cost.  Thus,  resource 
costs  and  military  worth  have  to  be  scrutinized  together. 

The  new  planning-programming-budgeting  procedure  facilitates 
the  performance  of  cost-effectiveness  studies  since  it  brings 
together  both  programs  and  costs  in  context  with  major  mili¬ 
tary  missions  of  the  Defense  Department  projected  over  a 
period  of  years." 

"For  example  the  RDT&E  subactivity  "Polaris  Submarines,"  for 
which  $380  million  was  included  in  the  1963  budget,  becomes 
part  of  the  program  element  "Polaris  System"  for  which  over 
$2  billion  was  included.  (This  in  turn)  is  part  of  the  group¬ 
ing  "Missile  Forces,  Sea  Based,"  which  in  turn  is  part  of  the 
major  program  "Strategic  Retaliatory  Forces." 

"The  key  point  for  fiscal  control,  for  both  budgeting  and 
programming  purposes,  is  the  RDT&E  subactivity.  In  the  case 
of  the  program  elements,  our  reporting  and  controls  are  being 
designed  primarily  to  relate  physical  performance  —  i.e., 
progress  in  achieving  the  objectives  of  each  program,  —  to 
total  cost  to  complete  the  development  and  investment  phases.. 
...and  the  annual  cost  of  operating  it.  The  really  important 
financial  question  in  making  decisions  about  the  Polaris 
system,  for  example,  .....is  not  how  much  the  program  will 
cost  during  any  one  budget  year,  but  how  much  it  will  cost 
to  complete." 

James  R.  Bridges,  Director  of  Electronics,  ODDR&E  ( 3 ) 

"We  mav  not  (be  giving  the  engineers)  enough  time  or  money  for 
thorough  design,  engineering  and  test.  In  proposals,  a  pros¬ 
pective  contractor ... .must  include  realistic  estimates  of  the 
time  and  cost  (to  satisfy  reliability  requirements).  We  (must) 
balance  the  cost  of  failure  against  the  true  value  of  early 
success . " 


1.2  NAVY  EXPERIENCE 

Hon.  Victor  M.  Longstreet,  Assistant  Secretary  of  the  Navy  for 
Financial  Management  (42) 

"Much  has  been  written  about  the  use  by  the  Defense  Department 
of  certain  techniques  as  a  basis  for  making  better  decisions. 
These  techniques  have  various  names  such  as  Cost-Effectiveness , 
Systems  Ef feet iveness ,... .the  purpose  of  which  is  to  take  a 
hard  look  at  everything,  to  explore  a  variety  of  possibilities, 
to  cane  up  with  facts,  to  apply  military  experience,  to  test, 
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to  evaluate,  to  weigh  judgments . One  very  important  in¬ 

gredient,  -  to  put  all  of  this  down  on  paper  for  all  to  see, 
to  study,  and  to  question." 

Rear  Admiral  Emerson  Fawkes,  USN,  Assistant  Chief,  BuWeps ,  R&D , 
Test  &  Evaluation  (4) 

"The  cost  of  acquisition  is  like  an  iceberg;  it  fails  to 
reveal  the  ownership  cost  of  4  to  10  times  the  acquisition 
cost . " 

"One  of  the  most  serious  of  our  problems  is  the  limited  ex¬ 
perienced  manpower  capability  to  which  the  Navy  has  access. 
This  in  turn  is  reflected  in  training  capabilities  required 
to  operate  and  maintain  these  complex  systems.  70%  of  main¬ 
tenance  is  done  by  "first  cruise"  sailors.  An  SRI  study 
indicates  that  direct  cost  per  first  class  electronic  tech¬ 
nician  is  $32,953  per  year." 

"The  use  of  cost/effectiveness  ratio  in  making  technical, 
management,  and  military  decisions  is  the  way  of  life.  Major 
decis ions .. .must  be  made  in  the  light  of  the  technical,  econ¬ 
omic,  and  military  "figures  of  merit." 

"We  must  obtain  a  major  advance  in  weapon  reliability  and 
maintainability.  It  is  here  that  the  greatest  cost  is  exper¬ 
ienced,  and  here  that  the  greatest  improvement  in  system 
effectiveness  can  be  obtained.  A  five  or  ten  percent  improve¬ 
ment  is  not  enough." 

"The  value  engineers  passion  for  simplicity  and  the  reliabil¬ 
ity  engineers  analytical  training  should  combine  to  attack 
our  spiralling  complexity  problem." 

Future  Trends  in  Carrier  Aviation,  US  Navy  survey  (5):  Figure 
26-6  shows  the  trend  of  complexity,  and  consequent  unreliability, 
upon  both  manpower  level  and  manhour  cost,  from  1941  to  1965. 

Note  particularly  the  period  1951  to  1960,  when  both  manpower 
level  and  manhours  (most  of  maintenance  cost)  doubled,  permitting 
the  inference  that  maintenance  time  was  not  improved. 

Figure  26-7  essentially  confirms  this.  It  shows  that  Operational 
Readiness,  or  Availability,  has  dropped  3  to  10%  in  the  same 
period. 

BIMRAB  (Bureau  (Weps)  Industry  Materiel  Reliability  Advisory 
Board)  Survey  on  Industry  Reliability  Programs  (6) s  This  20- 
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question  survey  by  BuWeps  in  the  Spring  of  1960  was  sent  to 
196  of  its  contractors.  Among  the  results  is  Figure  26-9.  Parti¬ 
cipants  were  asked  what  they  considered  to  be  the  deterrents 
limiting  reliability  to  present  state  of  the  art,  and  to  rank 
their  selections  from  1  to  5  in  order  of  importance. 

Clearly  the  contractor  consensus  is  that  funding  is  not  consistent 
with  the  manpower  necessary  to  get  the  desired  reliability.  In¬ 
cidentally  a  separate  Air  Force  survey  in  1961  to  72  contractors 
on  the  same  question  resulted  in  a  30.4%  funding  figure,  a  con¬ 
firmation  of  the  Navy  30.8%. 

1.3  AIR  FORCE  EXPERIENCE 

Lt.  General  Howell  M.  Estes,  March  1964  ( 7 ) 

"Maintenance  coats  on  today's  systems  have  risen  to... almost 
30%  of  the  Air  Force  budget.  Maintenance  of  military  elec¬ 
tronic  equipment  ranges  between  60  and  1000  times  the  initial 
costs.  Complexity  and  maintenance  creates  an  almost  insati¬ 
able  demand  for  large  quantities  of  highly  trained  manpower. 

The  progress  we  have  made  to  date  in  system  reliability .. .has 
simply  not  been  adequate  in  an  overall  sense." 

"Failure  of  a  $2  item  not  long  ago  caused  the  loss  of  a  $2.2 
million  launch  vehicle.  Ir.  another  program,  failure  of  a  $5 
thermal  shield  resulted  in  a  $23  million  disaster.  Failure  of 
e\  $25  fuel  valve  caused  loss  of  a  vehicle  and  damage  to  the 
site  totalling  $22  million.  These  dollars  would  have  bought 
a  lot  of  additional  reliability  in  these  programs." 

Air  Development  Center,  Research  Contract  {8):  The  anrual  support 
cost  for  a  representative  ground  communications  equipment  was 
about  12  times  the  original  cost.  For  navigation  it  was  6  times 
and  for  radar  0.6  times. 


2.  THE  COST-EFFECTIVENESS  BALANCE 

We  have  heard  rather  overwhelming  evidence  of  the  cost  of  main¬ 
tenance.  Maintenance  is  primarily  a  consequence  of  unreliability. 
We  have  heard  the  DOD  and  Navy  decision  to  put  quantitative  re¬ 
liability  and  maintainability  (R&M)  into  applicable  future  con¬ 
tracts.  But  it's  more  easily  said  than  done. 

The  fact  is  that  reliability  achievement  costs  money.  Improved 
engineering  and  quality  control  practices  often  improve  MTBF  by 
a  factor  of  2  to  4.  But  for  many  situations  we  need  10  or  100 
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times  improvement,  sometimes  1000.  Far  from  being  impossible, 
there  are  design  approaches  that  will  get  such  order  of  magnitude 
improvement,  as  outlined  in  chapter  33.  Eut  they  cost  money  for 
design  time,  perhaps  for  added  manufacturing  cost,  and  perhaps 
for  extensive  test  programs. 

So  we  are  in  the  position  of  having  to  spend  more  money  in  order 
to  save  money  in  inaintenar  _  ■ .  The  question  is  "how  much  relia¬ 
bility  and  maintainability  ;s  justified'*"  flow  m  *h  is  worth  the 
expenditure? 

2.1  THE  OBJECT IV E 


In  cider  tc  answer  this  question  we  have  to  establish  a  clear 
objective.  Is  the  objective  "maximum  reliability?"  Certainly 
not,  because  the  cost  of  achieving  it  could  far  offset  the 
maintenance  savings.  Is  it  "minimum  total  cost?"  Not  necessarily, 
because  often  a  moderate  cost  beyond  "minimum"  brings  substantial 
reliability  improvement,  and  in  turn  increased  system  effective¬ 
ness,  well  worth  the  extra  cost. 

What  we  are  really  after  is  maximum  system  accomp  1  ishment  or 
worth ,  in  relation  to  total  cost  to  acquire  and  maintain  the 
system  for  its  lifetime.  In  recent  years  this  has  come  to  be 
called  the  "cost-effectiveness"  of  the  system,  and  there  are 
three  common  approaches  to  its  opt imizat ion.  Those  are  design 
for  : 

(a)  The  maximum  ratio  of  effectiveness  to  total  cost 

(b)  The  maximum  effectiveness  for  a  g  j  v  e  n  t  o  t  a  .1  co  s  t  ,  and 

(c)  The  minimum  total  cost  for  a  g  a  en  r  e  gu i r ed  e  f  f e  c  t i v  e  n  e  s  s . 

These  approaches  do  not  always  lead  to  the  same  result,  which 
consideration  is  beyond  our  scope  here.  B  t  in  a  very  general 
way  we  can  say  that  we  always  want  to  get  (a)  t ho  most  effective¬ 
ness  per  dollar,  but  we  may  have  to  back  off  from  that  opt  imam 
to  (b)  live  within  available  funds,  and,  or  (c)  at  least  achieve 
some  minimum,  value  of  effectiveness. 

2.2  EFFECTIVENESS 

What,  actually,  do  we  mean  b\  Effectiveness?  The  term  is  still 
evolutionary,  and  many  conflicting  definitions  have  been  used. 

As  developed  in  chapter  25,  the  one  we  find  most  broadly  useful 
and  realistic  is  this: 


26-11 


Effectiveness  is  a  quantitative  index  expressing  actual 
accomplishment  or  worth  of  an  operational  system  or  component. 
It  is  a  function  of 

(a)  Performance  Capability, 

(b)  Delivery  Effectiveness, 

(c)  Reliability  and/or  Availability,  and 

(d)  Utilization 

It  may  be  the  simple  product  of  these  factors. 

Performance  Capability  is  a  quantitative  figure  of  merit  expres¬ 
sing  the  system  or  component  capability  of  performing  desired 
functions,  assuming  no  delivery  delay,  no  failure,  and  full 
ut i 1 1 z at  ion . 

Delivery  Effectiveness  is  the  ratio  of  system  or  component  effec¬ 
tiveness  as  degraded  by  late  delivery,  to  the  effectiveness  had 
it  been  available  when  neede  '  . 

Reliability  is  the  probability  that  the  system  or  component  will 
perform  its  intended  function  for  a  specified  period  under  stated 
condit ions . 

Avai lability  is  the  fraction  of  the  total  desired  operating  time 
than  the  system  or  component  is  operable. 

Uti lization  is  the  fraction  of  performance  capability  actually 
utilized  due  to  the  specific  application  and  environment  encoun¬ 
tered.  It  includes  all  effectiveness  d  gradation  due  to  causes 
external  to  the  system  or  component  itself. 

Thus  in  a  very  simplified  way  we  can  see  that  by  starting  with  a 
quantitative  index  c e  Performance  Capability,  and  multiplying  it 
by  realistic  "derating"  factors  (due  to  delivery  delay,  unrelia¬ 
bility,  downtime,  and  incomplete  utilization),  we  get  an  Effec¬ 
tiveness  figure  that  expresses  realistic  accomplishment  or  worth. 
Many  systems  require  something  more  complex  than  this  simple 
product,  but  it  serves  to  visualize  the  prot'em. 

2 . 1  TOTAL  COST 


Now  what  do  we  ttu  an  by  Total  cost"  of  a  system?  We  mean  all 
costs  for  the  useful  lifetime  of  the  system.  To  use  the  ''’OD 
terminology,  Tota 1  Cost  is  the  sum  of  "Acquisition  Cost"  for 
development  and  production,  and  "Ownership  Cost"  of  operation, 
maintenance,  and  consequence  of  failures. 


26-12 


Development  Cost  is  the  total  cost  of  operations  analysis  (during 
conceptual  phase),  system  design  (during  conceptual  and  definition 
phase),  hardware  design,  hardware  prototypes,  test,  evaluation, 
and  schedule  slippage  for  thirs  phase. 

Production  Cost  is  the  total  cost  for  quantity  procurement,  manu¬ 
facture,  installation,  tests,  training,  and  schedule  slippage 
for  this  phase. 

Operation  Cost  is  the  total  cost,  for  the  system  or  component 
lifetime,  of  those  personnel,  facilities,  utilities,  consumables, 
ard  special  inputs  required  for  operation,  excluding  those  for 
maintenance . 

Maintenance  Cost  is  the  total  cost,  for  the  system  or  component 
lifetime,  of  those  Personnel,  facilities,  spare  components, 
logistics,  and  diagnostic  aids  required  for  maintenance. 

Consequence  Cost  is  the  total  cost,  for  the  system  or  component 
lifetime,  generated  external  to  the  system  or  component  as  a 
consequence  of  its  failures.  ce  may  include  damage  or  loss 

of  other  systems  or  components  including  human  productivity. 

Now  it  will  be  apparent  that  these  costs  are  all  determined  by 
*-he  above  Effectiveness  factors.  For  example  Development  Cost 
is  a  primary  function  of  the  required  Performance  Capability, 
but  also  of  the  required  delivery,  and  the  required  Reliability 
and/or  Availability.  Maintenance  Cost  is  a  primary  function  of 
the  achieved  Reliability  and/or  Availability,  but  also  of  Util¬ 
ization,  and  to  a  Jesser  extent  the  others. 

2.4  SAMBA 

The  Bureau  of  Ships  has  established  a  broad  program  called  Sys¬ 
tems  Approach  to  Managing  BuShips  Acquisitions  (SAMBA).  Its 
objective  "is  to  make  significant  and  continuing  improvements 
in  the  management  of  all  aspects  of  the  system  acquisition  pro¬ 
cess.  Its  Annex  C,  Engineering  Considerations,  contains  sections 
4.2.1  on  Cost  of  Acquisition,  4.2.2  on  Cost  of  Ownership,  4.3  on 
Reliability/Cost  Relationships,  and  5.2  on  Reliability.  In  gen¬ 
eral  it  recognizes  the  identical  relationships  discussed  in  this 
chapter  26,  and  recommends  a  system  approach  such  as  we  have 
detailed  herein. 

2 . 5  EMEC 


The  Department  of  the  Navy  has  established  the  Electronics  Main- 


tenance  Engineering  Center  (EMEC)  to  (a)  review  new  equipment, 

(b)  monitor  fleet  expedience,  (c)  analyse  failure  data,  (d)  re¬ 
view  parts  support,  (e)  evaluate  training  effectiveness  and  man¬ 
power  deficiencies,  and  { f)  prosecute  corrective  action.  The 
Center  should  find  considerable  utility  in  the  techniques  to  be 
discussed  in  this  chapter,  as  well  as  several  others. 

3  .  TRADEOFF  ANALYSIS 

Having  nailed  down,  in  very  abbreviated  fashion,  what  we  mean  by 
Total  Cost  and  by  Effectiveness,  let  us  turn  to  the  relationships 
of  their  constituents,  and  constituent  impact  on  Cost-Effective¬ 
ness  . 

Figure  26-14  provides  a  broader  picture  of  the  overall  tradeoff 
and  cost  relationships.  At  the  left  we  note  Acquisition  Cost 
"top  of  an  iceberg,"  which  provides  the  funds  to  achieve  required 
Performance  Capability,  Delivery  Effectiveness,  Reliability  and 
Maintainability,  and  the  inevitable  added  cost  of  delivery  slip¬ 
page,  if  any. 

Ownership  Cost,  then,  is  the  sum  of  the  operational  costs  related 
to  Performance  Capability,  the  maintenance  and  logistic  costs 
generated  by  unReliability  and  minimized  by  Maintainability,  and 
the  Consequence  Costs  generated  by  failures,  delivery  slippage,  etc. 
Total  Cost  is  then  the  sum  of  Acquisition  and  Ownership  Costs, 
the  whole  iceberg. 

The  design  objective  is  always  to  get  the  most  Effectiveness  for 
the  least  Total  Cost,  which  is  rarely  possible,  because  they  rarely 
coincide.  But  it  is  not  only  possible  tut  imperative  to  design 
for  (a)  *-he  maximum  ratio  of  Effectiveness  to  Total  Cost,  as 
constrained  by  (b)  the  maximum  Effectiveness  for  a  given  Total 
Cost,  and/or  (c)  the  minimum  Total  Cost  for  a  given  Effectiveness. 

It  i3  not  necessary  to  have  a  comprehensive  "system  model"  or 
"reliability  model"  to  predict  Cost-Effectiveness,  except  to  the 
extent  that  such  models  may  (or  may  not)  be  needed  to  predict 
Performance  Capability,  Reliability,  Maintainability,  and  Delivery 
Effectiveness  ( from  Pert) .  It  does  not  have  to  be  complex. 

During  design,  at  any  level  of  the  system,  the  design  engineer 
is  constantly  making  decisions  between  alternatives.  Many  of 
these  decisions  involve  "tradeoff",  or  sacrificing  a  little  of 
one  attribute  to  get  enough  of  another.  In  particular,  the 
design  engineer  must  leamto  trade  off  Performance  Capability, 
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Reliability,  Maintainability,  and  Delivery  Effectiveness  with 
each  other,  as  showTi  by  the  heavy  arrows  in  Figure  26-14,  to  the 
extent  that  Effectiveness  is  thereby  improved  and/or  Total  Cost 
is  reduced. 

3.1  COST-EFFECTIVENESS  RATIO 

In  section  2.1  above  we  established  a  primary  objective  of  the 
maximum  ratio  of  Effectiveness  to  Total  Cost,  recognizing  that 
there  may  also  be  constraints  of  maximum  cost  and/or  minimum 
effectiveness.  In  order  to  simplify  the  picture  we  will  use  the 
ratio  henceforth: 


Cost-Effectiveness  C-E 


Effectiveness  E 
Total  Cost  Ct 


For  Effectiveness  we  can  use  the  .product  (9,10)  of  Performance 
Capability  P  ,  Delivery  Effectiveness  D  ,  Reliability  R  ,  and 
Utilization  U  .  For  total  cost  we  can  use  the  sum  of  Acquisition 
Cost  Ca  and  Ownership  Cost  Cu  .  So: 


Cost-Effectiveness  C-E 


PDRU 


ca  +  cu 


Keeping  in  mind  that  for  many  systems  Availability  A  (which 
accounts  for  Maintainability)  is  logical  in  place  of  Reliability 
R,  and  for  many  the  product  AR  is  logical,  we  can  use  this 
expression  to  visualize  the  tradeoffs  involved.  For  example  if, 
for  the  desired  Performance  Capability  P,  the  Delivery  Effective¬ 
ness  D  would  be  poor  (take  too  long) ,  the  Bureau  would  consider 
reduced  P  to  get  better  D  without  unduly  increasing  Acquisition 
Cost  Ca. 


Similarly  if  desired  Reliability  R  would  require  total  expendi¬ 
ture  (Ca  +  Cu)  exceeding  allocated  funds,  the  Bureau  might  con¬ 
sider  later  delivery  (lower  D)  to  get  more  design  and  test  time 
for  R,  or  fewer  functional  frills  (lower  P)  to  increase  the 
Reliability  R. 


In  this  course,  we  are  concerned  only  with  Reliability  and  Main¬ 
tainability,  and  implicitly  Availability.  The  above  cost-effec¬ 
tiveness  equation  is  a  very  powerful  tool  for  balancing  tradeoffs 
of  R  and  M  with  each  other  and  with  P,  D,  U,  Ca  and  Cu.  Such 
balancing  of  all  for  maximum  Cost-Effectiveness  must  be  done. 

In  fact  it  has  been  done  intuitively,  if  not  analytically,  for 
most  systems.  But  intuition  is  far  from  trustworthy  for  very 
complex  systems . 
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To  illustrate  the  analytical  use  of  this  tool,  we  will  "suboptim¬ 
ize."  That  is,  we  will  examine  the  tradeoffs  between  Reliability, 
Acquisition  Cost,  and  Ownership  Cost,  assuming  that  the  other 
tradeoffs  will  not  substantially  alter  the  result.  And  usually 
they  do  not.  To  do  this,  we  will"hold  still"  the  other  Effec¬ 
tiveness  factors  by  simply  letting  P  =  100,  D  =  1.0,  and  U  =  1.0, 
so  that  Reliability  becomes  a  measure  of  Effectiveness.  Then: 


Cost-Effectiveness  C-Er  =  — - — - 

r  ca  +  Cu 

Now  we  can  examine  the  inherent  effect  of  Reliability  upon  these 
costs  Ca  and  Cu,  and  hence  upon  Cost-Effectiveness.  We  will  do 
this  first  at  the  system  level,  then  getting  right  down  to  speci¬ 
fic  shipboard  equipment  experience  data. 

3.2  TYPICAL  SYSTEM 


Now  what  we  want  to  know  is  "How  much  reliability  and  maintain¬ 
ability  is  worth  the  expenditure?"  Since  the  need  for  maintain¬ 
ability  is  a  function  of  reliability,  it  becomes  logical  to  see 
what  happens  to  Cost-Effectiveness  as  reliability  is  varied, 
for  various  values  of  maintainability.  So  we  will  use  a  relia¬ 
bility  scale  across  the  bottom  of  Figure  26-17.  For  visualiza¬ 
tion,  as  well  as  simpler  arithmetic,  we  will  use  system  Mean 
Time  Between  Failures,  or  MTBF.  And  by  using  "Relative"  MTBF  =  1 
fo^  state-of-the-art  reliability  achievable  without  extra  Acqui¬ 
sition  Cost,  we  have  a  scale  that  permits  visualization  of  re¬ 
liability  improvement  needed. 

Using  the  data  from  an  actual  system,  graph  E  shows  that  relia¬ 
bility  would  increase  if  MTBF  were  increased  beyond  state  of 
the  art.  For  this  particular  system,  Effectiveness  is  actually 
proportional  to  the  product  AR  of  Availability  and  Reliability, 
so  taking  a  typical  Maintainability  (MTTR)  into  account  results 
in  the  curve  AR.  And  since  we  have  "fixed"  Performance  Capabil¬ 
ity,  Delivery  Effectiveness,  and  Utilization,  this  curve  AR  be¬ 
comes  a  measure  of  the  Effectiveness  E  of  the  system. 

Now  referring  to  the  Total  Cost  graph  Ct  we  note  that  the  Acqui¬ 
sition  Cost  of  the  system  rises  with  required  MTBF.  Just  how 
much  will  be  discussed  later,  but  it  commonly  results  from  tighter 
engineering,  supplier,  and  manufacturing  controls,  ani  from  design 
for  higher  reliability  as  discussed  in  chapter  13. 

As  MTBF  is  increased,  the  resultant  Ownership  Cost  drops  about  as 
shown.  This  results  from  simple  reduction  of  the  number  of 
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failures  and  hence  corrective  maintenance  manpower,  facilities, 
and  inventory  to  take  care  cf  them.  Then  we  can  add  these  two 
cost  curves  to  obtain  the  Total  Cost  curve,  which  shows  a  minimum 
at  relative  MTBF  of  6.  The  curve  says  that  spending  an  additional 
$1.7  million  for  Acquisition  would  have  saved  $4.2  million  in 
Ownership  Cost,  for  a  net  $2.5  million  saving.  At  the  same  time 
the  Effectiveness  (i.e.,  AR)  would  rise  from  40  to  85%,  a  ratio 
of  about  2-to-l. 

But  this  is  no.  the  optimum  reliability!  In  the  Cost-Effective¬ 
ness  graph  E/ct  we  have  plotted  the  ratio  of  Effectiveness  to 
Total  Cost,  taken  directly  from  the  two  graphs  below,  and  find 
the  peak  is  at  relative  MTBF  cf  about  14.  The  AR  curve  has 
shifted  the  peak  to  higher  MTEF .  This  is  very  typical.  This 
curve  says  it's  even  better  to  spend  $2.3  million  on  Acquisition, 
saving  $4.5  million  in  Ownership,  netting  $2.2  million  savings, 
but  getting  2.9  times  as  much  Effectiveness.  These  curves  are 
built  on  the  data  from  a  very  real  vehicle  system,  only  the 
Acquisition  Cost  line  being  estimated. 


But  what  about  Maintainability?  Suppose  we  invest  in  design  for 
one  tenth  the  Mean  Time  to  Restore.  In  Figure  26-19  the  solid 
curves  are  identical  to  Figure  26-17.  The  dashed  lines  show  the 
predicted  result  for  MTTR  F  10.  Note  that  (a)  the  cost-effective¬ 
ness  is  about  doubled,  compared  to  almost  3-to-l  for  MTBF  x  10, 

(b)  the  best  MTBF  is  then  >5  ;nstead  of  *14,  and  (c)  the  achiev¬ 
able  cost-effect iveness  is  then  1  css  than  if  MTTR  were  left  alone. 

Thus  while  these  conclusions  apply  only  to  this  specific  system, 
it  is  clear  that  a  family  of  such  curves  can  show  the  optimum 
combination  of  MTBF  and  MTTR  for  any  system. 

3.3  TYPICAL  COMPONENTS 

Now  let's  turn  to  hardware  equipment  and  parts  in  Figure  26-20. 
This  figure  is  identical  to  figure  26-17  except  that  we  have 
carried  the  relative  MTBF  scale  at  the  bottom  to  the  much  higher 
values  associated  with  component  MTBF.  They  are  so  very  much 
higher  than  the  typical  Mission  Time  and  typical  MTTR,  that  Re¬ 
liability  and  AR  are  practically  100%. 

Again  these  curves  are  from  real  equipment  and  t..is  time  the 
Acquisition  Cost  data  were  meticulously  calculated  for  3  points, 
and  one  of  these  confirmed..  We  note  the  identical  Total  Cost 
situation,  but  this  time  the  peak  of  Cost-Effectiveness  coincides 
with  minimum  Total  Cost,  because  the  Effectiveness  curve  is  flat. 
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This  occurs  quite  commonly  for  equipment  and  parts  cost-effective¬ 
ness  analysis,  so  there  is  seldom  need  to  plot  the  ratio  to 
locate  optimum  MTBF. 

Thus  we  see  that  it  may  be  economic  nonsense  to  insist  on  "maxi¬ 
mum"  reliability.  There  is  always  a  value  of  MTBF  for  a  part, 
equipment,  or  system  beyond  which  there  is  diminishing  advantage. 
But  note  too  that  these  Cost-Effectiveness  and  Total  Cost  curves 
tend  to  be  quite  broad,  which  means  that  it  cos+s  very  little 
more  to  get  2  or  3  tames  the  "optimum"  MTBF  if  it  is  that  signi¬ 
ficant  to  the  mission. 

3.4  OTHER  TECHNIQUES 


Although  cost-effectiveness  analysis  seems  by  far  the  most  power¬ 
ful  and  useful  tool  that  can  be  used  to  determine  optimum  relia¬ 
bility  and  maintainability,  it  is  by  no  means  the  only  one  that 
can  be  vised.  There  have  been  a  number  of  attacks  on  this  trade¬ 
off  problem.  A  few  are  cited  herewith  fc  r  reference  purposes, 
for  those  who  wish  to  "dig  deeper.” 


An  excellent  review  of  the  engineering  tradeoffs  is  given  in 
reference  (11).  Under  "internal  tradeoffs"  it  covers  system  per¬ 
formance  tradeoff,  reliabi lity/cost  tradeoff,  reliability/schedule 
i  -adeoff,  reliabi lity/ con fidence  tradeoff,  and  combined  tradeoffs. 
Ii  also  discusses  external  tradeoffs  and  tradeoffs  as  program 
management  aids. 


Whei  '  Availability  has  more  significance  than  Reliability,  as 
for  many  shipboard  situations,  a  Per formance/Avai labil ity/Cosi 
block  diagram  can  be  constructed  (12).  Each  block,  including 
redundancy,  is  assigned  a  Performance,  Availability  and  cost. 

The  1  inomial  theorem  is  used  to  calculate  the  probability  of 
each  clock  combination  being  in  service.  Then  the  effect  of  1% 
Availability  improvement  of  each  block  is  calculated,  to  determine 
jie  mo;,  t  economical  oppor  tuni  t  ies  for  improvement.  This  corres¬ 
ponds  to  study  of  the  slopes  of  Figures  26-17,  26-19,  and  26-20. 

1 'tat  cost  can  be  plotted  against  development  Lime,  using  a  curve 
for  each  design  alternative  having  fixed  Reliability  or  Availabil¬ 
ity  (13  p . 18)  . 


Standardization  of  parts,  equipment,  and  systems  not  only  reduces 
cost  but  increases  reliability  through  greater  refinement  of  the 
same  design.  The  tradeoff  Is  between  performance,  reliability, 
and  cost  (14). 
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The  BuShips  Design  Work  Study  Program  (15  p.4)  provides  a  "logical, 
systematic,  fact-finding  method  of  determining  what  needs  to  be 
done,  how  it  should  be  done,  and  who  should  do  it,"  in  finding 
more  economical  systems  of  men  and  equipment. 

System  reliability  vs,  weight  or  volume  tradeoff  curves  for 
various  levels  of  redundancy  may  be  used  (16  p.64,67). 

Industrial  main  boiler  forced  outage  (unavilability)  of  1%  i s 
said  to  require  4  to  5%  increase  of  reserve  capacity  to  conserve 
the  same  system  reliability  (18).  Variations  of  this  are  given 
in  reference  (19)  and  discussed  in  reference  (37, p. 135). 

3.5  PRACTICAL  ANALYSIS 

It  is  seldom  difficult  to  write  fairly  simple  equations  (ordinary 
algebra)  expressing  (a)  Effectiveness  in  terms  of  its  interrelated 
factors,  (b)  each  Effectiveness  factor  contribution  to  Total  Cost, 
and  then  (c)  the  expression  for  Effectiveness/Cost  ratio.  This 
can  and  always  should  be  done  as  a  prime  design  guide  for  trade¬ 
off  decisions. 

Cost-effectiveness  analysis  of  a  proposed  alternative  or  design 
change,  relative  to  a  first  alternative  or  current  design,  is 
very  useful.  By  evaluating  the  curve  slopes  in  Figures  26- 17, 26- in , 
and  26-20  rather  than  their  absolute  values,  the  dependency 
upon  uncertain  absolute  data  is  greatly  reduced  and  confidence 
in  the  result  can  be  very  good.  See  also  reference  (20, p.2 1 , 22 ) . 

Faced  with  a  choice  between  redundancy  and  component  improvement 
to  improve  reliability,  reference  (26)  provides  an  excellent 
analytical  tradeoff  approach  based  on  cost-effectiveness.  It 
incidentally  concludes  that  (a)  the  optimum  strategy  can  be  im¬ 
provement  of  the  strongest  instead  of  the  weakest  reliability 
link,  (b)  large  sums  spent  on  component  development  may  neve 
achieve  the  minimal  system  effectiveness,  and  (c)  redundant  major 
components  can  easily  achieve  the  minimal  system  effectiveness. 

Reliability  and  Maintainability  can  bs  mads  to  substitute  for 
•sch  other  to  some  degree,  over  a  wide  range  of  Availability 
(22,33  p.8-23).  Cost-effectiveness  analysis  can  be  used  to  make 
ths  tradeoff  economically  optimum. 

Value  Engineering  techniques  (23,24)  are  called  out  in  the  Guide 
for  ths  Preparation  of  TDP  Dependability  Plane  (25,  p.16).  The 
identical  techniques  can  be  used  with  cost-effectiveness  (instead 
of  simple  Acquisition  cost)  criteria  to  achieve  optimum  reliabil¬ 
ity  and  maintainability,  as  discussed  in  chapter  13,  section  1.1. 


4. 


EXAMPLES 


With  this  background  we  can  now  get  into  some  specific  examples 
and  data.  Such  published  lata  is  very  scarce.  We  have  scoured 
the  literature  and  found  a  few  marine  equipment  examples  and 
several  naval  electronic  equipment  examples.  But  most  of  the 
needed  data  can  be  obtained  by  going  after  it.  And  it  is  becom¬ 
ing  painfully  obvious  that  we  must  establish  channels  to  get, 
classify,  collate,  and  distribute  it  fairly  automatically. 

4.1  PUMP  TOTAL  COST  STUDY 

An  Alii-' -Chalmers  study  (27)  of  their  5"  x  4"  KSK  and  SK  pumps, 
used  by  the  Navy,  was  undertaken  for  pumps  sold  between  1953  and 
1962.  Failure  and  operating  time  information  on  117  Navy-owned 
pumps  was  obtained  from  aircraft  Carriers,  the  NAVSHIP-527  Machin¬ 
ery  History  Card,  and  the  NAVSHIPS-3621  Reports  of  Equipment 
Failure.  Also  104  responses  were  obtained  from  commercial  cus¬ 
tomers.  Cost  information  was  obtained  from  All is -Chalmers , 
adjusted  to  the  base  year  1954,  and  normalized  for  proprietary 
reasons.  Very  detailed  analysis  is  given  in  the  reference. 

Referring  to  Figure  26-24,  all  pumps  were  classified  by  design 
similarity  into  groups.  The  Acquisition  curve,  as  a  function  of 
MTBF  of  each  group,  is  the  manufacturers  normalized  selling  price, 
prorat ing  assumed  30,000— hour  life  to  the  fraction  of  price  for 
1000  hours.  It  includes  specifications,  R&D,  product  engineering, 
engineering  changes,  new  patterns,  small  tools,  burden,  shipping 
expense,  warranty  costs,  and  1%  profit. 

Preventive  Maintenance  costs  for  parts  and  labor  were  calculated 
on  the  assumption  that  standard  recommendations  were  followed. 

Corrective  Maintenance  costs  for  parts  and  labor  were  calculated 
from  the  accumulated  failure  data,  all  of  which  were  factored  to 
common  seawater  operation.  The  figures  alonside  the  curve  show 
number  of  group  failures  on  which  the  point  is  based,  as  an  in¬ 
dication  of  confidence  level  surrounding  the  MTBF  value.  Note 
the  large  number  of  73  for  one  group. 

The  Total  curve  adds  the  three  below  it.  Clearly  the  trend  is 
reduced  cost  for  higher  MTBF.  We  can  also  tenatively  conclude 
that  optimum  MTBF  is  off  the  chart  at  still  higher  MTBF.  In  any 
event  it  is  apparent  that  increased  Acquisition  Cost  of  high  re¬ 
liability,  within  this  range  at  least,  pays  off  several-fold  in 
reduced  Total  cost. 
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If  we  know  the  Mean  Time  to  Restore,  which  the  reference  does 
not  give,  we  could  calculate  pump  Availability.  Then  we  could 
draw  a  Cost-Ef fectivoness  curve  using  the  ratio  of  Availability 
to  Total  Cost.  But  since  pump  Availability  is  probably  99%  or 
better,  the  Cost-Effectiveness  curve  would  be  essentially  a  re¬ 
ciprocal  of  the  Total  Cost  curve,  leading  to  identical  conclu¬ 
sions.  Thus  we  see  that  this  step  is  useful  only  when.  MTTR  or 
Mission  Time  are  substantial  in  relation  to  MTBF. 


4.2  TURBINE  STOP -VALVE  TOTAL  COST 

The  following  example  is  a  projection  of  existing  steam 
system  design  to  conceptual  redundant  design  alternatives.  It 
uses  existing  valve  MTBF  and  cost  data  to  predict  the  achievable 
MTBF  and  cost  of  alternative  untried  configurations.  It  is  not 
verification  of  achieved  Cost-Ef fectiveness ,  but  does  illustrate 
the  use  of  cost-effectiveness  analysis  in  design  to  obtain  the 
optimum  reliability. 

The  stop  valve  of  a  public  utility  turbo-generator  (29)  is  con¬ 
trolled  by  the  overspeed  governor,  and  is  used  only  when  it  ex¬ 
ceeds  speed  by  10%.  The  valve  is  expected  to  close  completely 
within  0.3  seconds  when  the  generator  loses  its  load,  or  for 
other  reasons.  Failure  to  close  may  cause  "blowing”  of  the 
turbogenerator,  and  consequently  destruction  of  the  equipment. 

Five  stop  valve  arrangements  are  shown  at  the  top  of  Figure  26-26, 
the  horizontal  lines  indicating  the  input  manifold  to  the  govern¬ 
ing  valves,  which  in  turn  feed  the  turbine.  In  system  A  the 
failure  of  either  valve  to  close  will  fail  to  stop  steam  flow. 
However  either  valve  may  be  exercised  in  periods  of  light  load, 
(say  every  week)  to  insure  satisfactory  operation,  because  the 
other  will  sustain  turbine  speed.  In  system  B  the  valve  may  not 
be  exercised  until  shutdown  (say  every  4  months).  Thus  the 
shorter  "mission"  time  (1  week)  results  in  higher  reliability 
for  A  than  b,  even  though  failure  of  either  valve  is  system 
failure. 

Using  the  data  collected  on  valves  with  operating  time  totalling 
over  3  million  hours,  analyses  of  the  reliability  of  5  such  stoo 
valve  arrangements  has  been  computed,  as  indicated  at  ABCDE  on 
the  horizontal  scale. 

Acquisition  Costs  for  A  and  B  were  obtained  from  existing  arrange- 
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ments ,  C,  D  S:  E  being  computed  using  the  component  costs  of 
valves,  piping  supports,  equalizing  passages,  etc.  For  propri¬ 
etary  reasons  the  cost  results  are  normalized  u^ing  an  arbitrarv 
but  fixed  factor. 

Ownership  Costs  include  Maintenance  and  Cons equence-of- failure 
costs.  Maintenance  includes  scheduled  preventive  maintenance 
every  3  years,  as  well  as  unscheduled  corrective  maintenance 
based  upon  separate  operational  and  non-operational  failure 
rates  and  the  reconditioning  and  restoration-to-service  cost  each 
time . 

Consequence  Costs  occur  when  stop  valve  failure  has  resulted  in 
turbo- generator  damage  beyond  repair.  For  a  100-MW  unit  it  may 
take  2  years  to  get  another  unless  one  was  being  manufactured, 
during  which  time  an  older  less-efficient  one  may  be  used  or 
power  bought  from  another  utility.  Such  costs  were  estimated  at 
150%  of  the  turbo-generator  Acquisition  cost,  plus  5%  of  original 
dollar  output  at  75%  capacity.  All  costs  were  calculated  over 
30-year  life.  Similar  estimates  can  be  made  for  shipboard  failure 
in  terms  of  the  manpower  and  depreciation  cost  for  added  mission 
time  at  reduced  performance. 

Total  Cost  is  then  the  sum  of  Acquisition  and  Ownership  costs,  as 
plotted  in  Figure  26-26.  It  is  clear  that,  of  these  alternatives, 
minimum  Acquisition  Cost  at  B  does  not  provide  minimum  Total  cost- 
It  ir,  also  clear  that  there  is  an  optimum  stop-valve  system  re¬ 
liability,  achievable  via  system  D.  Higher  reliability  would  be 
less  economical. 

The  Effectiveness  of  a  system  is  its  accomplish  ment  of  objectives, 
which  has  not  been  taken  into  account  in  the  above.  But  even  if 
we  assume  the  worst  possible  Reliability  (MTBF  =  6.8  x  106  hours) 
and  Maintainability  (inability  to  get  a  turbo-generator  replace¬ 
ment  for  two  years  or  17500  hours)  the  Availability  is 
6.8  x  106/(6.8  x  106  +  17500  hours)  or  99.3%.  For  the  utility 
this  is  a  measure  of  Effectiveness,  since  Performance  Capability, 
Delivery,  and  Utilization  are  unchanged.  If  we  plotted  Cost- 
Effectiveness  (the  ratio  of  Effectiveness  to  Total  Cost)  against 
MTBF  in  the  above  example  we  would  gain  no  further  insight. 

For  a  Navy  ship  it  is  not  uncommon  to  have  a  fourth  750  KW  turbo¬ 
generator  for  reliability  insurance,  where  thi.ee  can  actually 
handle  peak  loads,  and  two  can  nicely  handle  normal  loads.  Thus 
Effectiveness  is  not  degraded  by  one  failure.  But  the  question 
does  arise  whether  the  added  reliability  of  the  fourth  is  worth 
its  cost.  An  analysis  similar  to  the  above  could  be  conducted# 
taking  battle  environment  into  account  as  a.  potential  cause  of  failure 
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4.3  G UILANCE  COMPUTER 


Early  in  1959  IBM  was  awarded  a  contract  for  a  missile  guidance 
computer  (30)  with  an  MTBF  requirement  50  times  that  being  re¬ 
alized  in  an  operational  bombing  navigational  system.  This  re¬ 
quirement  was  met  on  schedule  because  the  reliability  activities 
were  a  prime  factor  in  R&D  planning. 

Three  different  proposals  were  generated,  each  for  a  different 
MTBF  level.  The  anticipated  costs  for  these  are  shown  in  Figure 
2o  29,  in  which  cost  and  MTBF  have  been  normalized  for  proprietary 
and  security  reasons.  On  the  horizontal  scale,  unity  is  the  MTBF 
achievable  with  a  ''normal"  reliability  program.  The  other  two 
proposals  were  for  4  and  10  times  this.  The  decision  was  for  4, 
which  is  50  times  the  0.08  previously  obtained.  The  actual  re¬ 
sult  obtained  was  6.8,  via  the  detailed  t'chniques  detailed  in 
the  reference. 

Figure  26-30  shows  the  actual  reliability  program  costs  as  per¬ 
cent  of  contractors  progrant  cost.  Such  figures  are  commonly 
used  as  indices  of  reliability  effort,  but  are  also  very  tricky 
because  of  the  wide  disparity  in  meaning  of  the  words  among  con¬ 
tractors.  Yet  "reliability  programs"  have  ranged  5  to  15%, 
sometimes  higher.  The  reference  contains  descriptions  of  the 
detailed  activities. 

The  reference  then  gives  detailed  attention  to  operational  cost 
breakdown  for  a  quantity  of  50  computers.  The  Figure  26-31 
"Production  and  Operating"  cost  curve  combines  these  quantity- 
related  costs.  The  "Development"  cost  curve  is  repeated,  so  that 
the  two  may  be  added  to  get  the  "Total"  cost  curve.  It  will  be 
seen  that  minimum  total  cost  occurs  at  around  3  to  4  times  MTBF 
improvement,  while  the  actual  turned  out  6.8  for  somewhat  higher 
cost. 

In  the  absence  of  actual  MTBF  and  mission  time  data,  we  can  com¬ 
pute  the  reliability  for  mission  times  of  1,  0.1,  and  0.01  of 
the  state-of-the-art  MTBF  of  1.  If  Performance  Capability, 
Delivery  Effectiveness,  and  Utilization  are  unity,  and  there  is 
no  opportunity  for  maintenance,  then  Effectiveness  is  measured 
by  Reliability.  So  we  can  divide  Reliability  by  Total  Cost  to 
get  Cost-Effectiveness.  We  see  that  optimum  MTBF  has  moved  up 
slightly  for  mission  time  of  0.1,  but  quite  significantly  for 
1.0  mission  time. 

For  1.0  mission  time  we  see  that  optimum  MTBF  improvement  is 
about  7  times,  which  is  what  the  program  actually  achieved.  Had* 
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we  looked  only  at  minimum  Total  Cost,  wo  v.ouio  have  eluded 
that  about  3  -  r>  was  ”nf imum,  and  that  the  af't’.i  ;i  nob.  i  women  t  was 
excess ive . 

4 . 4  POLA2  IS  C,  LIT  D  ANC  F.  COM  PUT  ER  MAINT  FNANCF  •’ 

A  Study  (31)  was  undertaken  of  the  best  overall  ma  i  ni  .-n  a  net- 
strategy  for  period  Leu  l  lv- checked  e-mipcnt  which  t\.nrv,  *■  be  re¬ 
supplied  in  its  opet  at  ion  a!  environment ,  such  as  a  suhm.ari.no 
cruise.  Relatively  Mttle  time  was  spent  on  data  gathering ,  so 
that  the  results  should  be  regarded  as  :  1  1  us t r  ■  1 1  re  more  than 
conclus ive . 

The  analytical  model  considers  a  t  only  "out  of  pocket"  costs 
such  as  manpower,  repair  facilities,  test  eouiprunt,  etc.,  but 
also  "real"  costs  such  as  those  due  to  shortage  of  replacement 
parts  and  modules,  and  submarine  space.  The  inclusion,  of  shor¬ 
tage  cost  prevents  minimization  of  cost  at  the  expense  of  opera¬ 
tional  Effectiveness.  Costs  were  computed  vs.  MT3F  on  the  basis 
of  four  possible  maintenance  policies,  the  fifth  below  being 
essentially  the  same  as  the  fourth : 

1.  Discard  and  salvage 

2.  Repair  on  th_  submarine 

3.  Repair  on  the  tender 

4.  Repair  at  .he  factory 

5.  Repair  at  the  Naval  Weapons  Annex 

The  cost  factors  considered  were: 

1.  Inventory  support  costs 

2.  Parts  for  repair 

3.  Test  and  repair  equipment  and  facilities 

4 .  Submar i n e  spar o 

q .  Restoration  manpower 

b .  Trans  port  at  ion,  hand l i ng  and  pack ng i ng . 

figure  26-33  shows  the  effect  of  maintenance  locale  upon  Main¬ 
tenance  ( there  fore  Ownership)  Cost ,  the  "Discard"  curve  being 
the  cost  of  replacing  without  repair  of  fail*  d  undo.  N-d  sur¬ 
prisingly,  t  he  discard  of  an  entire  computer  is  n  >t  ocomwn  l  ca  1 
except  at  very  high  MTBF.  Repair  on  the  tender  turns  >ut  best, 
with  submarine  a  close  second. 

A  compar  ison  between  integrated  and  plug-in  computer  r;n  in 
Figure  26-34,  assuming  tender  maintenance  for  both.,  shows  the 
great  importance  of  this  dos ign  decision.  F  r  unity  rel at i\ e 


ANNUAL  TENDER  MAINTENANCE  COST 

PoLaris  Guidance  Computer 
for  14  submarines 
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MT8F  (which  may  be  state  of  the  art)  the  plug-in  design  saves 
$800,000  annually,  mainly  through  reduction  of  required  inven¬ 
tories.  On  the  other  hand  plua-in  designs  tend  to  ha^e  higher 
Acquisition  Cost  (for  design  and  manufacture)  and  because  of 
their  higher  number  of  connections  tend  to  have  lower  MTBF.  But 
$800,000  would  buy  a  32-man  continuous  design  effort  to  overcome 
inherent  plug-in  reliability  problems  and  achieve  order ~of-mag~ 
nitude  MTBF  improvement. 

5 .  RELIABILITY  ACQUISITION  COST 

We  have  discussed  at  some  length  an  analytical  approach  to  design 
tradeoffs.  But  like  most  analytical  techniques,  it  requires 
meaningful  data  inputs  in  order  to  draw  meaningful  conclusions. 
This  section  discusses  one  very  important  input,  Acquisition  Cost 
data,  perhaps  the  most  difficult. 

5.1  _ ACQUISITION  COST  ELEMENTS 

The  cost  of  achieving  reliability  is  very  real,  but  such  cost 
data  is  very  scarce.  To  understand  the  sources  of  such  cost  we 
can  consider  four  broad  categories  as  illustrated  in  Figure  26-36. 

1.  Traditional  design  practices  where  the  design  engineer  draws 
upon  his  personal  experience,  uses  much  intuitive  judgment, 
and  conducts  some  analysis.  The  product  is  built,  its  fail¬ 
ures  are  analysed,  _t  is  redesigned,  and  at  least  several 
such  complete  cycles  are  required  before  the  product  achieves 
maturity.  (In  shipbuilding  many  such  cycles  have  long  since 
occurred,  resulting  in  standard  "margin"  practices)  But 
today  we  often  need  adequate  reliability  the  first  time  in 
designs  for  which  no  experience  is  available,  and  cannot 
tolerate  the  time  and  cost  of  recycling. 

2.  Modern  design  practices  are  employed,  using  standard  parts 
control,  supplier  control,  configuration  control,  education 
in  new  technologies,  data  dissemination  in  convenient  form, 
system  design  and  analysis,  stress/strength  analysis,  and 
design  review.  Those  practices  get  moderately  good  relia¬ 
bility  in  areas  where  there  is  little  prior  experience,  and 
are  expected  as  part  of  any  substantial  contract  without 
additional  cost. 

3.  Reliability  programs  add  reliability  program  planning,  "re¬ 
liable"  design  and  reliability  evaluation  techniques,  relia¬ 
bility  education,  quantitative  reliability  requirements  on 


Traditional  Design  Practices 
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designers  and  suppliers,  analysis,  test  and  audit  of  design, 
stress  time  and  failure  recording  and  data  dissemination, 
failure  analysis,  corrective  action  control,  and  the  use  of 
MIL  R  38100  "Established  Reliability"  parts.  Such  programs 
get  steady  reliability  growth,  typically  2-to-l  MTBF  improve¬ 
ment  in  4  years . 

4.  Design  to  specified  reliability  does  not  mean  design  to  a 

"goal",  merely  using  "modern  design  practices"  and  a  "relia¬ 
bility  program"  to  get  the  ’  oest  possible"  reliability  with¬ 
in  contracted  cost. 

It  does  mean  conscious  design  to  a  speci fied  quantitative 
MTBF  or  %  reliability,  no  mere  and  no  less,  which  is  rarely 
done.  It  involves  simplification  techniques,  standardiza¬ 
tion,  parts  selection  and  application,  s tress/strength  design, 
tolerance  evaluation,  failure  rate  prediction,  human  engin¬ 
eering,  failure  cause  and  effect  avoidance,  preventive  main¬ 
tenance  provision,  proaucibility ,  supplier  evaluation  and 
control,  evaluation  tests,  local  environment  control,  failure 
prediction  devices,  component  integration,  redundancy,  and 
parts  improvement.  See  chapter  13. 

Such  techniques  can  often  get  one  or  more  orders  of  magnitude 
reliability  improvement  to  satisfy  the  actual  need.  They 
can  make  some  of  the  tight  "control"  elements  in  3  above 
unnecessary. 

Most  contractors  do  not  have  real  "Reliability  Programs",  in 
the  above  sense,  but  the  good  ones  utilize  the  above  Traditional 
and  Modern  practices.  Of  the  contractors  who  do  have  "reliabil¬ 
ity  programs",  almost  all  have  some  fraction  or  all  of  the  above 
reliability  program  elements.  Such  reliability  programs  gener¬ 
ally  cost  5  to  10%  of  the  contract  design  cost,  excluding  factory 
quality  assurance,  inspection,  and  test. 

Design  to  specified  reliability  has  been  undertaken  by  relatively 
few  contractors.  The  techniques  are  spotty,  not  really  shaken 
down,  and  the  reliability  required  by  such  specification  is  sel¬ 
dom  achieved  due  to  contractual  cost  constraints.  Yet  the  tech¬ 
niques  are  imperative  for  many  of  our  projected  long-life  appli¬ 
cations  like  submarine  missions,  underwater  dormant  weapons, 
space  probes,  etc.  They  may  be  costl"',  yet  perhaps  no  more  so 
than  the  reliability  "control"  techniques,  some  of  which  would 
then  not  be  needed.  But  the  payoff  in  maintenance  cost  reduction 
above  justifies  much  more  than  is  spent  today. 
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Design  for  best  cost-effectiveness  is  design  fcr  whatever  value 
of  Reliability  and  Maintainability  is  best  considering  Total 
Cost.  It  will  generally  cost  more  for  Acquisition,  but  Ownership 
Cost  savings  result  in  a  net  Total  Cost  reduction. 

Figure  26-36  thus  attempts  to  show  that  even  though  “good”  relia¬ 
bility  is  being  obtained  with  and  without  heavy  reliability 
"control"  programs,  such  programs  are  often  merely  a  crutch  for 
obsolete  or  immature  engineering  practices.  Whan  this  is  the 
case,  it  should  be  possible  to  get  much  better  reliability  through 
better  engineering  practices,  and  smaller  reliability  programs, 
without  increasing  the  Acquisition  Cost.  But  to  actually  get  the 
optimal  reliability  for  best  cost-effectiveness,  still  higher 
Acquisition  Cost  may  be  needed. 

5.2  ACQUISITION  COST  HISTORY 


In  Figure  26-39  we  have  converted  several  items  of  Acquisition- 
Cost  vs.  reliability  data  to  a  common  basis  for  comparison. 

Relative  MTBF  of  unity  indicates  state  of  the  art.  Relative  $ 

Cost  of  unity  is  the  Acquisition  Cost  (purchase  priced  at  state 
of  the  art.  More  details  on  tlie  background  fcr  each  curve  are 
obtainable  from  the  references  indicated  below. 

The  Pump  curve  results  from  merely  historical  information  (27) 
over  nine  years,  without  benefit  of  a  reliability  program  as  out¬ 
lined  above.  The  Guidance  Computer  curve  comes  fr oia  a  detailed 
plan  of  three  alternative  approaches  (30)  from  light  to  heavy 
reliability  programs.  The  Stop  Valve  Redundancy  curve  actually 
results  from  redundant  use  (29)  of  one  to  four  identical  valves 
in  different  configurations,  an  excellent  example;  of  this 
technique. 

The  Navigation  Unit  curve  is  based  on  a  factual  comparison  (32  ,j3) 
of  total  cost  of  redesign  and  production  to  original  cost  of 
design  and  production,  which  was  10%  cheaper  for  6-t.o-l  MTBF 
improvement.  But  in  the  absence  of  design  cost  data  we  have 
assumed  that  original  design  cost  25%  of  the  total,  so  that  fresh 
design  and  production  would  have  cost  15%  more  than  the  original 
design  and  production. 

The  Launch  Vehicle  Estimate  curve  resulted  from  a  consideration 
(10)  of  the  limits  between  which  it  could  possibly  range,  then 

trial,  intuitive  judgment,  and  correction  of  several  alternatives 

5.3  ACQUISITION  COST  DATA  SOURCES 


RELIABILITY  ACQUISITION  COST 


*e  Of  RELATIVE  MTBF 
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As  indicated  above,  data  relating  design  and  manufacturing  cost 
to  MTBF  achieved  is  very  scarce.  The  data  presented  above 
resulted  from  several  weeks  research  of  the  past  few  years’  relia¬ 
bility  conference  and  periodical  literature,  with  emphasis  on 
Navy  material.  Certainly  more  could  be  turned  up  with  further 
detailed  research. 

When  adequate  component  cost/MTBF  data  cannot  be  found  in  the 
literature,  the  following  approaches  will  often  get  it: 

1.  Informal  request  of  manufacturer 

2.  Formal  request  for  manufacturers  quotatio;'.  at  several 
■\7Z7  IpvpIs, 

3.  Informal  request  of  contractors  who  have  used  the  component. 

4.  Bureau  procurement  records  for  cost  and  maintenance  records 
for  MTBF. 

When  direct  cost/MTBF  data  is  not  obtainable,  one  must  resort  to 
indirect  approaches.  Each  kind  of  design  and  manufacture  presents 
its  own  problems,  but  generally  the  analysis  approaches  are: 

1.  Gross  evaluation  of  cost/MTBF  a  hieved  by  very  similar 
or  identical  sequential  programs. 

2.  Synthesis  of  incremental  task  c'ists  to  achieve  various 
MTBF  levels. 

3.  Calculation  of  cost  and  MTBF  achievable  with  various 
redundancy  configurations  of  components  of  known  cost  and 
MTBF.  This  is  a  "worst  case"  cost  which  actual  design  should 
reduce. 

4.  Analysis  of  cost  and  MUF  as  related  to  a  complexity  common 
denominator,  using  data  for  real  "des  gn  for  reliability"  pro¬ 
grams  . 

5.  Analysis  of  manufacturers  aintenarce  schedule  and  spare 
parts  recommendations. 

6.  OWNERSHIP  COST 


This  element  of  Total  Cost  is  the  aggregate  of  all  costs  to  the 
user  after  the  system  is  first  made  operational,  and  incurred 
throughout  its  useful  life.  It  is  also  commonly  called  "user" 
cost,  and  sometimes  "operational"  cost.  It  is  most  of  the  iceberg. 
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The  Operational  Cost  elements  (training,  operator  salaries  and 
facilities,  etc.,  for  operators)  are  fairly  easily  predictable 
from  study  of  the  system  design  itself. 

The  Maintenance  Cost  elements  (preventive  and  corrective  main¬ 
tenance,  maintenance  personnel  salaries  and  facilities,  spare 
parts,  logistics,  etc.)  typically  constitute  the  bulk  of  owner¬ 
ship  cost;  they  depend  primarily  upon  reliability  (MTBF)  and  to 
r  lesser  extent  upon  maintainability  (MTTR) . 

The  Consequence  Cost  dements  (damage  to  or  loss  of  equipment, 
pataonnei,  or  other  resources  external  to  the  system  as  a  result 
of  it"  -v.-4  -'—I*-  -’-p — *  J  ’y  -  p  in  how  the  system  is 

used,  and  upon  its  reliability  (MTBF)  and  safety.  But  Consequence 
costs  and  maintenance  costs  are  almost  the  sole  cd^crd  of  the 
importance  of  reliability,  and  hence  justification  for  its 
achievement . 

Ownership  Cost  data  must  come  from  the  owner  or  user,  as  opposed 
to  Acquisition  Cost  data  from  the  Contractor  or  manufacturer.  But 
it  is  just  as  scarce,  for  different  reason.  While  a  growing  amount 
of  field  maintenance  data  is  being  collected,  it  is  as  yet  by  no 
means  adequate  for  many  specific  Ownership  Cost  analyses.  The 
analyst  must  resort  to  indirect  methods  based  on  available  data. 

NAVSHIPS  94324  Maintainability  Design  Criteria  Handbook  (34) 
provides  detailed  maintenance  times  for  each  increment  of  electronic 
maintenance,  as  well  as  skill  levels  available.  These  can  be  con¬ 
verted  to  direr*  maintenance  costs. 

The  Machinery  History  Card,  NAVSHIPS  527  (27  p„175)  is  an  important 
source  of  maintenance  manpower  data.  Reports  of  Equipment  Failure 
NAVSHIPS  3621  (27  p.176)  also  provide  maintenance  manpower  data. 
Shipyard  repair  records  (35  p.344)  contain  much  maintenance  cost 
data. 

Many  papers  have  presented  methods  of  obtaining  or  estimating 
ownership  costs.  Each  kind  of  system  d. sign  and  utilization 
presents  its  own  problems,  but  generally  the  analysis  approaches 
are ; 


1.  Gross  evaluation  of  cos t/MTBF/MTTR  achieved  by  very  similar 
or  identical  sequential  programs. 


2.  Synthesis  of  incremental  task  costs  resulting  from  various 
MTBF  or  MTTR  levels  and  environments. 


3.  Straightforward  collection  of  user  data  during  operation, 
which  of  course  is  too  late  for  the  current  design  but  may  be 
quite  valuable  for  the  next. 

7 .  OPPORTUNITIES  FOR  IMPROVEMENT 

The  principles  of  cost-effectiveness  analysis  are  not  being  applied 
to  design  except  in  very  rare  instances.  But  this  does  not  detract 
from  their  significance  ard  the  imperative  need  to  apply  them  as 
quickly  as  we  can.  Mr,  McNamara  has  directed  their  broad  appli¬ 
cation,  and  Messrs.  Hitch,  Secretary  Longs treet,  and  Rear  Admiral 
Fawkes  have  recommended  their  use  wherever  feasible. 

This  section  addresses  most  of  the  reasons  that  the  principles 
have  not  yet  been  applied,  and  translates  each  to  deliberately 
concise  statements  of  programs  that  the  Bureau  could  and  should 
consider  i;.it  1  -  .c  action  or.  cost-e f fectiveness  analysis. 

7.1  PROPOSAL  REQUIREMENTS 

Require  selected  prospective  contractors  to  include  in  their  pro¬ 
posals  an  analysis  of  total  costs  and  effectiveness  vs.  a  range 
cf  .  »TBF  and  MTTR . 

7.2  COST-EFFECTIVENESS  STUDY 

When  the  problem  is  too  complex  for  meaningful  analysis  in  pro¬ 
posals,  issue  advance  study  contracts  to  say  three  experienced 
contractors,  each  to  conduct  a  cost-effectiveness  study  based  on 
his  own  development  and  manufacturing  costs-  This  will  shc-w  what 
MTBF  and  MTTR  is  achievable,  and  tell  much  about  the  contractors 
reliability  and  maintainability  capability  (27  ,  p.  2. 19,  item  ,2). 
Recently  the  Bureau  has  been  directed  by  OOP  to  do  this  on  specific 
programs . 

7.3  ACQUISITION  COST  DATA  IN  CONTRACTS 

Require  contractors  to  provide  development  and  manufacturing  cost 
data  by  principal  component,  whether  or  not  it  may  also  be  needed, 
by  department.  (27  p.214  item  5;  p . 2 1 5  item  lb)-  Then  relate  to 
actually  achieved  MTBF  and  MTTR. 

7.4  ACQUISITION  AND  OWNERSHIP  COST  DATA  STUDIES 

Where  data  is  not  available. on  completed  contracts,  but  the  data 
is  sorely  needed  for  decisions  on  optimal  MTBF  and  MTTR  on  future 
contracts,  award  study  contracts  to  collect  rhe  data.  Several  of 
these  have  been  concluded  by  BuShips  and  BuWeps .  Another  alter¬ 
native  is  to  require  a  ne*  design  contractor  to  do  this  first, 
as  a  basis  for  his  design. 
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7.5  OWNERSHIP  COST  DATA  PROGRAMS 


Establish  s.ich  activities  as  the  Navy  Maintenance  and  Materiel 
Management  Project  Group  (36)  and  the  BuShips  Maintenance  Manage¬ 
ment  Project  Office  (37)  to  obtain  the  cost  data  needed  for  MTBF 
and  MTTR  d^c^sions.  Vigorously  pursue  full  documentation  of 
reliability,  cost,  and  maintenance  data  during  the  life  of  the 
equipment,  and  make  it  available  to  the  manufacturer  for  his 
analysis  and  action  (27  p.217  item  4). 

7.6  REFERENCE  MANUALS 

Contract  for  the  development  of  a  very  concise  reference 
manual  containing  not  theory  and  analysis  but  the  significant 
fact  figures,  and  quantitative  "rules  of  thumb"  for  d ii ect  use  by 
contractor  design  engineers.  An  example  of  a  "rule  of  thumb"  is 
"Doubling  the  MTBF  adds  about  20%  to  Acquisition  Cost". 

7.7  TRADEOFt  xvuiio 


Contract  for  the  development  of  new  and  more  refined  tradeoff 
tools  relating  design  alternatives  to  cost.  Simple  research  will 
go  far  toward  refinement  of  the  above  20%  with  variations  for 
equipment  categories,  as  well  as  toward  realization  of  similar 
tools  for  ownership  cost  vs.  MTBF  and  MTTR  (27  p.214  item  6; 
p.216  item  13;  P-219  item  3;  38  p.7). 

7.8  CONTRACT  TECHNIQUES 


Develop  realistic  incentive  contract  techniques  that  encourage 
contractor  expenditure  up  to  that  which  achieves  the  optimal  MTBF 
and  MTTR,  considering  total  cosc. 

Consider  what  happens  on  a  straight  cost-incentive  contract  that 
pays  say  10%  of  Acquisition  cost  savings.  If  the  centi  ,ct  target 
is  state-of-the-art  reliability  and  maintainability,  it  is  to  the 
contractor's  advantage  to  spend  as  little  as  possible  on  relia¬ 
bility  improvement  as  long  as  he  thinks  he  knows  how  to  design 
state-of-the-art  goods.  But  he  knows  that  he  will  be  penalized 
for  any  extra  expenditure  to  achieve  better  reliability  or  main¬ 
tainability.  The  same  is  true  of  "value  engineering"  (39)  clauses, 
which  do  not  yet  take  into  account  ultimate  customer  savings. 

Consider  what  happens  on  a  reliability-incentive  contract  thau  oays 
say  0.3%  added  fee  for  each  %  operational  re’iability  improvement 
tr  addition  to  the  above  cost-incentive.  The  cost  of  achieving 
2%  higher  reliability  is  much  greater  at,  say,  90%  than  it  is 
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at  70%.  Therefore  the  m-.re  reliability  a  contractor  achieves  the 
less  he  is  paid  for  it.  There  is  always  a  reliability  level  be¬ 
yond  which  the  contractor  is  actually  penalized  for  reliability 
improvement  (38  p.9;  27  p.217  it  '5).  For  the  above  figures  (10) 
it  is  around  90%  reliability. 

Perhaps  you  will  say  "if  this  is  the  situation  for  a  certain 
system,  then  the  user  should  specify  exactly  what  MTBF  he  requires.” 
Indeed  he  should,  but  usually  cannot.  He  cannot  because  today  he 
does  not  know  the  cost  to  a ch i ev e  various  levels  of  reliability  and 
maintainability.  The  Bureau  will  usually  have  to  obtain  these  costs 
from  contractors  in  order  to  locate  the  crest  of  the  cost-effective¬ 
ness  curve.  This  should  occur  roughly  for  the  LaA,  and  later  more 
precisely  in  competitive  dialogue  with  contractors.  Such  costs 
will  vary  widely  between  contractors. 

Another  much-discussed  solution  is  to  give  the  contractors  pert¬ 
inent  maintenance  and  logistic  parameters  in  several  study  con¬ 
tracts,  and  let  them  submit  cost-effectiveness  analyses  based  on 
th“ir  own  costs.  With  such  analyses  the?  Bureau  should  be  able 
to  locate  the  crest,  ar.d  incidentally  find  out  who  knows  what  it 
costs  to  achieve,  not  just  "contr  1",  reliability  and  maintain¬ 
ability. 

Still  another  may  be  to  simply  award  contracts  on  the  basis  of 
promised  cost-effectiveness  instead  of  cost,  with  incentives  and 
penalties  surrounding  the  promised  value.  This  would  oblige 
contractors  to  develop  and  use  the  kind  of  cost-effectiveness 
analysis  tools  outlined  herein:  and  also  to  operate  a  cont '  n,'ous 
cost-effectiveness  model  incorporating  Reliability  and  Maintain¬ 
ability  prediction.  Unlike  reliability  models,  these  would  predict 
loss  or  gain  of  fee,  guaranteed  to  get  action  in  proportion  to  the 
real  utlima^e  impact  on  the  user. 

As  discussed  (40)  in  detail  by  Dr.  Hitch,  DOD  Comptroller,  the 
problem  of  properly  responsive  incentives  is  very  complex.  Like 
yachting  handicaps,  incentive  contracts  invite  design  to  the  payo f f 
formula.  The  priolerr.  is  to  make  the  formula  exact  ly  match  the  rt  il 
BuShips  objective,  which  is  not  usually  as  simple  as  the  ratio  of 
effectiveness  to  cost.  But  it  does  seem  as  though  the  potential 
paycfi  to  the  Bureau  and  taxpayers  is  worth  considerable  effort  on 
one  or  more  solutions  to  the  problem. 

7.9  MANAGEMENT  VISIBILITY 

Develop  a  realistic  monthly  reporting  system,  for  BuShips  and 
contractor  management  alike,  that  tells  on  one  page  whether  the 
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anticipated  operational  cost-offoctiv'-ness  is  v  * up  or  down. 
One  example  is  shown  in  Figure  -  If  a  cost  ly  change  w- . 

made  to  obtain  higher  MTBF but  the  expected  ma  inter--.  _  saving 
more  than  offsets  it.,  then  the  report  can  show  the  trend  line 
going  to  higher  Cost-Effectiveness.  Or  if  a  test  shows  lowcr- 


than-planned  MTBF,  the  trend  depression  shows  how  much  can  be 
afforded  to  correct  it  ( 38  p.ll;  4i  p.127). 


7.10  DATA  COLLECTION  STUDY 


Augment  the  current  study  to  develop  more  effective  and  efficient 
failure  reporting,  data  feedback,  and  corrective  action  procedures 
(see  Chapter  9  and  references  27  p.219,  42  p.356,  and  28  p.4)  to 
include  failure  cost  reporting  by  the  Bureau.  By  calculating  in¬ 
dices  at  regular  intervals,  codin',  and  storing:  for  easy  retrieval , 
the  time  and  cost  of  many  other  studies  should  be  drastically  re¬ 
duced.  Enough  data  should  be  collected  to  determine  which  com¬ 
ponents  need  further  development,  based  on  Total  Cost  (27  p.216  item  9; 


7.11  SURVEYS 


For  critical  systems,  establish  separate  personnel,  trained  in 
the  criteria  for  and  applications  of  the  data,  to  obtain  Owner¬ 
ship  Cost,  manpower,  MTBF  and  MTTR  data.  Experience  has  shown 
beyond  all  doubt  that  recording  by  the  using  and  maintenance 
personnel  is  not  reliable  (44  p.29). 


7.12  NAVY  MACHINERY  HISTORY  CARD 


When  properly  tilled  in,  this  provides  very  useful  data.  But  often 
the  very  important  hours  of  operation  are  not  given  (27  p.214  item 
5;  p . 2 1 7  item  9).  One  report  (44  p.65)  states  that  the  Navy  failure 
report ing  program  is  exceptional  m  concept ,  but  has  not  been  getting 
failures  reported.  At  present  only  10  to  20*  reporting  is  achieved 
(27  p.217  item  10;  4  3  p.4  item  r>).  The  date  the  equipment  was  put 
into  operation  should  be  entered,  as  well  as  the  exact  observation 
that  led  to  maintenance  action  (2?  p.2i?  item  9).  While  this 
card  is  not  transmitted  to  the  Bureau,  it  could  provide  excellent 
da* a  for  analysis  of  specific  components. 


7.13  MA INT FINANCE  SCHEDULES 

fhe  preventive  and  correct. . e  maintenance  schedules  furnished  by 
the  contractor  should  be  analytically  worked  out  and  based  upon 
the  failure  rate  curve  and  all  costs  involved  (2-  p.tl  >  item  6,7). 


MONTHLY  COST-EFFECTIVENESS  REPORT 
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8 ,  SUMMARY 

In  this  lecture  ve  have  shown  that  the  cost  consequences  of 
inadequate  reliability  and  maintainability  are  very  substantial. 
They  constitute  a  major  drain  on  the  defense  dollar  resources, 
which  dollars  might  be  better  spent  on  other  procurement. 

Every  decision  between  design  alternatives  should  be  evaluated 
on  the  basis  of  consequent  system  Effectiveness  in  relation  to 
consequent  Total  Cose  of  Acqusition  and  Ownership.  Delivery 
effects  are  included  in  Effectiveness  and  Costs.  This  is  virtually 
the  only  sound  basis  upon  which  Reliability  and  Maintainability 
reauirements  can  be  established,  and  concurrently  justified. 

Comparative  analysis  between  alternatives  requires  data  only  on 
things  that  change  between  alternatives,  thus  works  well  with 
limited  data.  Absolute  prediction  of  Cost-Effectiveness  requires 
much  more  data. 

Reliability  and  Maintainability  Acquisition  Cost  data  must  be 
obtained  from  contractors  as  part  of  proposals  and  refined  during 
design  and  production.  Ownership  Cost  data  must  be  obtained  from 
fleet  operations  and  shipyards. 

Contracts  can  be  written  so  as  to  provide  cost-effectiveness  data, 
and  eventually  should  be  wuritten  in  such  a  way  that  the  contractors 
fee  is  directly  related  to  achieved  cost-effectiveness. 

As  Rear  Admiral  Emerson  Fawkes,  USN,  said,  "The  use  of  the  cost- 
effectiveness  ratio  in  making  technical,  management,  and  military 
decisions  is  the  way  of  life." 
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Chapter  27 


DEFINITIONS 

Until  the  reader  or  listener  sees  rigorous  technical  definitions 
for  common  words  used  in  a  technical  sense,  he  takes  them  to  have 
the  "common  usage"  meaning.  But  since  very  few  readers  actually 
see  standardized  technical  definitions,  adequate  communication 
demands  that  technical  definitions  be  consistent  with,  and  fall 
within,  the  dictionary  and  common  usage  meanings.  To  do  other¬ 
wise  leads  the  average  reader  astray,  cutting  off  the  very  under¬ 
standing  that  is  sought. 

This  problem  commonly  occurs  in  new  technologies ,  such  as  relia¬ 
bility  and  maintainability.  MIL  STD  721A  and  MIL  STD  778  have 
been  issued  in  an  attempt  to  standardize  terms  commonly  used  in 
these  technologies.  But  new  technologies  are  dynamic,  and  the 
definitions  that  worked  yesterday  are  often  found  inadequate  to¬ 
morrow.  Thus  many  of  the  72  lA  and  778  definitions  need  improve¬ 
ment  for  communciation  with  design  engineers  in  todays  technology. 

In  this  chapter,  therefore,  we  list  selected  definitions  of  words 
as  used  in  this  text.  Wherever  a  721A  or  778  definition  is 
adequate,  we  use  it.  Where  it  is  not,  we  provide  a  workable 
definition  followed  by  an  indented  quotation  of  72lA  or  778  and 
an  explanation  of  its  limitations. 

Al  so  some  additional  terms  actually  needed  and  used  in  the  advan¬ 
cing  technology  are  defined.  The  following  definitions  are  grouped 
according  to  their  relationship  to  each  other,  rather  than  alpha¬ 
betically. 

1.  RELIABILITY  DEFINITIONS 

1.1  FAILURE  is  the  inability  of  the  system  or  component  to 
perform  the  required  function.  As  defined  in  6.2  and  6.10  below, 
"system"  and  "component"  are  used  in  a  broad  sense,  and  specifically 
include  human  components. 

MIL  STD  72lA  says  "Failure  is  the  inability  of  materiel  to 
perform  its  required  function  within  previously-established 
limits."  "Materiel"  means  "hardware"  at  all  levels,  specifi¬ 
cally  excluding  human  components  wh^se  failure  must  be  in¬ 
cluded.  "Previously  established  limits”  may  apply  to  testing 
but  not  necessarily  to  the  actual  operational  limits. 
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1.2  OPERATING  TIME  is  the  time  during  which  the  system  or  com¬ 
ponent  is  performing  its  intended  function. 

MIL  STD  7 2 lA  is  identical  except  materiel  is  used  in  place  cf 
system  or  component,  but  again  there  must  be  no  exclusion  of 
human  components. 

Operating  time  is  quite  significant  for  many  purposes,  but 
often  cannot  be  related  directly  to  failures.  To  obtain 
failure  rate  or  MTBF  it  is  necessary  to  use  Stress  Time, 
defined  below. 

1.3  STRESS  TLME  is  the  time  during  which  stresses  occur  that 
can  induce  failure.  It  includes  Operating  Time.  Such  stresses 
commonly  occur  during  standby  and  maintenance. 

1.4  MISSION  TIME  (t)  is  the  period  of  time  in  which  an  item 
must  perform  a  spe'-i  fi_d  mission.  (MIL  STD  77S) . 

In  the  above,  "item"  includes  systems  and  components. 

"Specified  mission"  normally  means  the  specific  task  whose 
completion  without  failure  is  required. 

1.5  MEAN  TIME  BETWEEN  FAILURES  (MTBF)  (T)  is  the  average  Stress 
Time  between  Failures.  (See  Stress  Time). 

MIL  STD  721a  says  "MTBF  is,  for  a  particular  interval,  the 
total  measured  functioning  time  (or  cycles,  miles,  events, 
etc.)  of  a  population  of  materiel  divided  by  the  total  number 
of  failures  within  the  population  during  the  measured  period". 
Again  it  cannot  be  limited  to  "materiel"  and  the  time  may  be 
more  than  "functioning"  time.  MTBF  is  the  primary  index  of 
design  reliability,  commonly  expressed  in  hours.  It  is  the 
reciprocal  of  Failure  Rate. 

1.6  MFAN  CYCLES  BETWEEN  FAILURES  (MCBF )  is  the  average  number 
of  stress  cycles  between  failures.  An  operational  Stress  Time/ 
cycles  relationship  may  be  used  t<  convert  to  equivalent  MTBF 
for  analysis. 

1 .7  _ FAIL'JRE  RATE  (\),  at  any  point  in  the  life  of  the  system 

or  component,  is  the  incremental  change  j.n  the  number  of  failures 
per  associated  incremental  change  in  the  measure  of  life  (time, 
cycles,  mi 'os,  events,  c*c.,  as  applicable)."  When  failure  rate 
is  assumed  constant,  it  is  the  average  number  of  failures  per 
unit  Stress  Time.  It  is  commonly  expressed  in  failures  per 
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million  hours,  and  is  the  reciprocal  of  Mean  Time  Between  Failures 

MIL  STD  72lA  is  identical  to  the  primary  statement  above, 
except  that  "materiel"  excludes  human  components. 

1.8  _ WEAROUT  FAILURE  is  one  that  occurs  as  a  result  of  deterior¬ 

ation  processes  or  mechanical  wear  and  whose  probability  of  occur¬ 
rence  increases  with  time  (MIL  STD  721a) . 

1.9  RELIABILITY  (R)  is  the  probability  that  the  system  or 
component  will  perform  its  intended  function  for  a  specified 
period  under  stated  conditions. 

MIL  STD  721a  is  identical  except  that  "materiel"  excludes 
human  components.  The  word  is  also  commonly  used  to  express 
the  fraction  of  systems  or  components  that  operate  without 
failure  for  the  Mission  Time  duration. 

1.10  CONFIDENCE  LEVEL  is  the  probability  that  a  given  statement 
is  correct,  or  the  chance  that  the  true  value  lies  between  two 
confidence  limits  (the  confidence  interval)  (MIL  STD  721a) . 

The  commonest  use  of  the  term  is  the  probability  that  the  true 
value  of  reliability  is  at  least  equal  to  a  specified  lower 
limit. 

2.  MAINTAINABILITY  DEFINITIONS 


Figure  27-6  will  be  found  helpful  in  understanding  the  Maintain¬ 
ability  and  Availability  definitions. 

■yi 

2 . 1  DOWNTIME  is  that  portion  of  calendar  time  during  which  th§ 
item  is  not  in  condition  to  perform  its  intended  function  (MIL 
STD  778) . 

"Item"  refers  to  systems  or  components.  Note  that  downtiidfe 
is  maintenance  time,  but  excludes  any  during  which  the  system 
or  component  continues  to  operate,  as  well  as  any  that  can  be 
instantly  interrupted. 

2.2  PREVENTIVE  MAINTENANCE  TIME  is  the  maintenance  time  to 
retain  an  item  in  satisfactory  operational  condition  by  providing 
systematic  inspection,  detection,  and  prevention  of  incipient 
failure.  It  is  made  up  of  performance  measureir. evt,  care  of 
mechanical  wearout  items,  front  panel  adjustment,  calibration 
and  alignment,  cleaning,  etc.  (MIL  STD  778). 


AVAILABILITY  TIME  CHART 
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2 . 3  CORRECTIVE  MAINTENANCE  TIME  is  the  tir  e  that  begins  with 
'e  observance  of  a  malfunction  of  an  item  and  ends  when  the 
item  is  restored  to  a  satisfactory  operating  condition.  It  may 
be  subdivided  into  Active  Maintenance  Time  and  Non-Active  Main¬ 
tenance  Time  (.MIL  STD  7'?3). 


This  is  also  called  "Repair  Time"  by  the  Navy  and  "downtime" 
or  "unscheduled  maintenance  time"  by  many  industries.  To 
avoid  confusion  in  this  text  we  have  tried  to  avoid  use  of 
these  alternative  words  for  this  meaning. 


2.4  ACTIVE  MAINTENANCE  TIME  is  the  time  during 
and  corrective  maintenance  work  is  actually  being 
(MIL  STD  778) 


which  preventive 
done  on  the  item 


2.5  NON- ACTIVE  MAINTENANCE  TIME  is  the  time  during  which  no 
maintenance  is  being  accomplished  on  the  item  because  of  either 
supply  or  administrative  reasons  (MIL  STD  778) 

“Supply  or  administrative  reasons"  means  waiting  for  any  item, 
cr  any  other  reason. 

2.6  ACTIVE  RESTORATION  TIME  is  the  Corrective  Maintenance  Time 
during  which  work  is  actually  being  done.  It  includes  detection, 
diagnosis,  preparation,  replacement  or  repair,  adjustment,  check¬ 
out,  and  reload  time  to  the  extent  each  is  necessary. 

MIL  STD  "’78  says  "active  REPAIR  time  is  the  time  during  which 
one  or  more  technicians  are  working  on  the  item  to  effect  a 
repair."  It  is  felt  that  to  design  engineers  the  word  "repair 
implies  only  one  of  many  steps  in  the  restoration  after  a 
failure. 


2.7  MEAN  TIME  TO  RESTORE  (MTTR)  is  the  statistical  mean  of  the 
distribution  of  times-to-restore .  It  is  the  summation  of  active 
restoration  times  during  a  given  period  of  time  divided  by  the 
total  number  of  failures  during  the  same  time  interval. 

MIL  STD  778  is  identical  except  for  the  word  ’repair"  instead 
of  "restoration",  which  implies  only  a  portion  of  the  task. 

MTTR  is  the  mean  time  for  restoration  to  full  performance 
capability,  including  detection,  diagnosis,  preparation, 
replacement  or  repair,  adjustment,  checkout,  and  (for  loss  of 
content)  reload,  and  any  waiting  for  replacements,  instructions 
test  equipment,  etc. 
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2 .8 _ MAINTAINABILITY  is  the  speed  or  economy  with  which  a 

system  or  component  can  be  kept  in,  and/or  restored  to,  full 
performance  capability.  A  principally-used  measure  is  the 
average  number  of  failures  restored  per  hour  of  Corrective  Main¬ 
tenance  time,  which  is  the  reciprocal  of  MTTR .  Another  is  the 
fraction  of  attempts  wherein  restoration  is  completed  in  a 
specified  time,  or  the  probability  that  it  will  be  completed  in 
that  time.  Another  is  the  functional  time  obtained  per  dollar 
cost  of  preventive  and  corrective  maintenance. 

MIL  STD  778  says  "maintainability  is  a  characteristic  of 
design  and  installation  which  is  expressed  as  the  probability 
that  an  item  will  conform  to  specified  conditions  within  a 
given  period  of  time  when  maintenance  action  is  performed  in 
accordance  with  prescribed  procedures  and  resources."  Un¬ 
fortunately  this  wording  seems  to  include  predicted  relia¬ 
bility  and  availability  within  maintainability  and  the  words 
"probability",  "will",  "specified",  "given",  and  "prescribed" 
preclude  use  of  the  word  "maintainability"  to  express  what 
actually  happens  in  operation. 

3.  AVAILABILITY  DEFINITIONS 


3 . 1  AVAILABILITY  (A)  is  the  fraction  of  the  total  desired 
operating  time  that  the  system  or  component  is  operable. 

MIL  STD  7 2 lA  is  identical  to  the  above,  except  that  the  word 
"materiel"  excludes  human  components.  MIL  STD  778  defines 
the  same  thing  as  "availability  (operational)  is  the  prob¬ 
ability  tha*  a  system  or  equipment  when  used  under  stated 
conditions  and  in  an  actual  supply  environment  shall  operate 
satisfactorily  at  any  given  time." 

3 . 2  DEMAND  AVAILABILITY  (Aj)  is  the  fraction  of  required  Stress 
Time  that  the  system  or  component  can  perform  its  function  upon 
demand.  It  may  be  expressed,  as  MTBF/(MTBF  +  MTTR)  when  there  is 
no  stress  during  maintenance,  but  as  1  -  MTTR/MTBF  when  there  is 
stress  during  maintenance.  It  is  the  Availability  assuming  that 
Preventive  Maintenance  can  be  interrupted  or  will  be  done  in 
Inactive  Time. 

3.3  CONTINUOUS  AVAILABILITY  (Ac)  is  the  fraction  of  long-term 
Stress  Time  that  the  system  or  component  can  perform  its  function, 
with  preventive  maintenance  accomplished.  One  expression  for  it 
i*  1  -  MTTR/MTBF  -  Tp/MTBF  where  Tp  is  the  preventive  maintenance 
time  ratio  to  number  of  failures. 
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3 . 4 _ OPERATIONAL  READINESS  is  the  fraction  of  total  number  of 

systems  or  components  that  are  in  condition  to  perform,  their 
function.  It  is  usually  equal  to  Continuous  Availability. 

4 '  EFFECTIVENESS  DEFINITIONS 

Figure  27-11  may  help  to  visualize  the  relationships  between 
effectiveness  factors,  cost  element;,  and  cost-effectiveness, 

4.1  _ EFFECTIVENESS  (E)  is  a  quantitative  index  expressing 

accomplishment  or  worth  of  an  operational  system  or  component. 

It  is  a  function  of  performance  capability,  delivery  effective¬ 
ness,  availability  and/or  reliability,  and  utilization.  It  may 
oe  the  simple  product  of  these  factors. 

MIL  STD  721a  says  "effectiveness  is  the  probability  that  the 
materiel  will  operate  successfully  when  required."  Unfor¬ 
tunately  this  definition  is  not  distinguishable  from  relia¬ 
bility,  and  does  not  sufficiently  identify  the  constituents. 

4.2  SYSTEM  EFFECT IVENESS  is  the  Effectiveness  of  a  system. 

4.3  PERFORMANCE  CAPABILITY  (P)  is  a  quantitative  figure  of 
merit  expressing  the  system  or  component  capability  of  performing 
desired  functions,  assuming  no  delivery  delay,  no  failures,  and 
full  utilization. 

4.4  DELIVERY  EFFECTIVENESS  (D)  is  the  ratio  of  system  or  com¬ 

ponent  effectiveness  as  degraded  by  late  delivery,  to  the  effec¬ 
tiveness  had  it  been  available  when  needed.  It  is  also  sometimes 
called  Schedule  Effectiveness  or  Schedule  Adherence. 

4. f>  UTILIZATION  ( U )  is  the  fraction  of  performance  capability 

actually  utilized  due  to  the  specific  application  and  environment 
encountered.  It  includes  all  effectiveness  degradation  due  to 
causes  external  to  the  system  or  component  itself. 

4.6 _ USEFUL  LIFE  (L)  is  the  system  or  component  life,  in  year*, 

as  limited  by  obsolescence,  utility,  and  wearout . 

5 ,  COST  DEFINITIONS 

5  1  DEVELOPMENT  COST  (C^)  is  the  total  cost  of  operations 
analysis  {during  conceptual  phase),  uesign  (during  concep¬ 
tual  and  definition  phase),  hardware  hardware  prototypes, 

test,  evaluation,  and  schedule  slippage  for  this  phase. 


THE  ELEMENTS  OF  COST-EFFECTIVENESS 


27-11 


5 . 2  _ PRODUCTION  COST  (Cp)  is  the  total  cost  for  quantity  procure- 

men4-,  manufacture,  installation,  tests,  training,  and  schedule 
slippage  for  this  pnase. 

5 . 3  _ OPERATION  COST  (CQ)  is  the  total  cost,  for  the  system  or 

component  lifetime,  of  those  personnel,  facilities,  utilities, 
consumables,  and  special  inputs  required  for  operation,  excluding 
those  for  maintenance. 

5 . 4  _ MAINTENANCE  COST  (Cm)  is  the  total  cost,  for  the  system  or 

component  ,i ft  time,  of  those  personnel,  facilities,  spare  com¬ 
ponents,  logistics,  and  diagnostic  aids  required  for  maintenance. 

5.5  CONSEQUENCE  COST  (Cc)  is  the  total  cost,  for  the  system  or 
component  lifetime,  generated  external  to  the  system  or  component 
as  a  consequence  of  its  failures.  These  may  include  damage  or 
loss  of  other  systems  or  components,  including  human  productivity. 

5.6  _ ACQUISITION  COST  (Ca)  is  the  total  cost  for  Development 

(C<j)  and  Production  (Cp)  defined  above. 

5.7  _ OWNERSHIP  COST  (also  called  User  Cost,  Cu)  is  the  total 

cost  for  Operation  (Cc) ,  Maintenance  (Cm) ,  and  Consequence  (Cc) 
defined  above. 


5.8  TOTAL  COST  (Ct)  is  the  total  cost  for  Acquis  it  ion  (Ca )  and 
Ownership  (Cu)  defined  above. 

5.9  COST-EFFECTIVENESS  is  the  actual  quantitative  accomplish¬ 
ment  or  worth  of  an  operational  system  or  component,  relative  to 
its  Total  Cost,  taking  delivery  time  into  account.  it  may  be 
expressed  as  a  ratio  of  Effectiveness  (including  delivery  effective¬ 
ness)  to  Total  Cost  (including  late  delivery  costs).  Where 
Effectiveness  can  be  expressed  as  Worth  in  dollars,  Cost-Effective 
ness  can  be  very  significantly  expressed  as  Effectiveness  minus 
Total  Cost,  or  Net  Gain  or  "profit". 

6 .  GENERAL  DEFINITIONS 

6 . 1  _ ACCELERATED  LIFE  TEST  is  a  test  at  excessive  stress  or 

environment  to  reduce  test  time,  and  implies  adequate  correlation 
to  normal  stress  life. 

6.2  COMPONENT  is  a  constituent  of  a  higher- level  system  or  com¬ 
ponent.  It  usually  means  a  functional  hardware  assembly  at  any 
level  between  Parts  and  Systems,  but  can  include  human  components 
of  systems. 
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6.3  CRITICAL  COMPONENTS  are  those  whose  reliability  and  appli¬ 
cation  are  such  that  they  require  special  attention  to  preserve 
system  reliability.  The  criterion  for  such  designation  may  be  a 
Criticality  (defined  below)  above  a  specified  level. 

An  older  criterion  is  that  they  are  used  in  such  a  way  that 
their  failure  would  cause  system  failure,  which  criterion 
does  not  take  the  component  reliability  into  account. 

6.4  CRITICALITY  of  a  component  is  its  quantitative  contribution, 
relative  to  all  other  components,  to  predicted  system  failure  rate. 
A  CRITICALITY  RANKING  is  a  list  of  components  in  the  order  of 
their  decreasing  probability  of  causing  system  failure.  Thus 
criticality  depends  upon  both  component  failure  rate  and  the  way 

it  is  used. 

6.5  _ LIFE  TEST  is  any  test  at  simulated  normal  operating  stress 

and  environment  from  which  failure  vs.  stress  time  data  is  derived. 
Examples  are  sequential  life  tests,  "Agree"  tests,  MTBF  tests, 

etc . 


6.6  OVERSTPESS  TEST  is  any  test  in  which  stress  and/or  environ¬ 
ment  is  made  progressively  more  severe  until  either  failure  occurs 
or  adequate  safety  margin  is  demonstrated. 

6/7 _ PART  is  the  lowest- level  component  of  an  assembly,  not 

usually  subject  to  further  disassembly. 

6.8  _ REDUNDANCY  is  the  existence  of  more  than  one  means  for 

accomplishing  a  given  function  (MIL  STD  72 lA) . 

6.9  _ STRENGTH/STRESS  ANALYSIS  is  the  comparison  of  strength  dis¬ 

tribution  with  anticipated  stress  distribution,  to  determine 
safety  margin  or  the  orobability  of  no  failure  of  a  "population". 

6.10  SYSTEM  is  used  primarily  to  mean  the  overall  man-machine 
complex  to  accomplish  the  desired  functions.  But  it  also  is  used 
to  mean  a  group  of  components  that  work  together  to  accomplish  the 
functions.  Examples  are  weapon  system,  propulsion  system,  or 
hydraulic  system. 

6.11  VALUE  ENG  INKER IMG  is  the  determination  of  alternative 
means  of  accomplishing  required  component  functions  at  lower  cost, 
without  degradation  of  performance  capability,  reliability  or 
maintainability , 
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6.12  VERIFICATION  (of  K  &  M)  is  the  estimation  of  achieved  reliability  or 
maintainability  by  accumulation  of  factory  test,  life  test,  overstress  test, 
or  operational  data.  Analytical  prediction  and  stress  margin  analysis  do  not 
actually  verify,  but  may  provide  the  only  achievable  assurance  in  many  cases. 
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